MBML improves multi-task RL by inferring task identity from state-action pairs.
problem Multi-task batch reinforcement learning with unseen tasks.
method MBML uses triplet loss and relabeling to robustify task inference.
result Significantly faster convergence on unseen tasks compared to random initialization.
Structural identity is a concept of symmetry in which network nodes are identified according to the network structure and their relationship to other nodes. Structural identity has been studied in theory and practice over the past decades, but only recently has it been addressed with representational learning technique…
RLINK uses deep reinforcement learning to improve user identity linkage across social networks.
problem Recognizing the same user across different social networks.
method Converts user identity linkage into a sequence decision problem and uses deep reinforcement learning to optimize the linkage strategy.
result Achieves better performance than state-of-the-art methods in experiments on various datasets.
Three scenarios for continual learning tasks are described and compared.
problem Difficulty in comparing continual learning methods due to different evaluation protocols.
method Three continual learning scenarios based on task identity and inference requirements.
result Different scenarios have varying difficulty and require different approaches.
Most existing word embedding approaches do not distinguish the same words in different contexts, therefore ignoring their contextual meanings. As a result, the learned embeddings of these words are usually a mixture of multiple meanings. In this paper, we acknowledge multiple identities of the same word in different co…
TAME learns new tasks without knowing them, outperforming existing methods.
problem Learning new tasks without task identities.
method TAME detects task shifts using loss function deviations and employs online pruning.
result TAME outperforms existing methods on continual learning benchmarks.
Bayesian weight priors improve neural network learning of identity relations.
problem Neural networks struggle to learn abstract and systematic relations, especially identity relations.
method Extended RBP approach using Bayesian weight priors as a regularization term.
result Bayesian weight priors lead to perfect generalization for identity relations and do not hinder standard neural network learning.
Augments GNNs with diversification to preserve node identity.
problem Current GNNs filter node information, potentially losing node identity.
method Integrates diversification operators with aggregation to enrich node representations.
result Significant performance boost on 9 node classification tasks.
New ITID assumption improves generalization in practical tasks.
problem Generalization theories fail to interpret many generalization phenomena and guide practical learning tasks due to the IID assumption.
method Proposes ITID assumption to consider task properties, derives a new generalization bound, and introduces an invariance enhancement algorithm.
result Demonstrates the effectiveness of the new generalization theory in improving practical generalization performance.
The paper provides guarantees for learning nonlinear representations from multiple non-identically distributed data sources.
problem Learning from non-identically distributed and dependent data.
method Established statistical guarantees for learning general nonlinear representations from multiple data sources.
result The excess risk of the estimated function decays as a function of the sample complexity and task diversity.
Unsupervised mesh disentanglement separates identity and pose.
problem Geometric disentanglement for 3D deformable models.
method CFAN-VAE architecture using conformal factor and normal features.
result CFAN-VAE achieves state-of-the-art performance on unsupervised geometric disentanglement.
OCEAN infers online task identities from context variables.
problem Online task inference for compositional tasks with context adaptation.
method Variational inference framework OCEAN models global and local context variables in a joint latent space.
result OCEAN provides more effective task inference with sequential context adaptation.
SupSup model learns thousands of tasks without forgetting, using randomly initialized subnetworks.
problem Sequentially learning many tasks without forgetting.
method Randomly initialized base network with task-specific subnetworks (supermasks).
result Gradient-based optimization can identify the correct subnetwork for new tasks.
Novel framework for data sharing and coordinated exploration in concurrent RL with non-identical environments.
problem Learning more data-efficient and better policies in concurrent RL with non-identical environments.
method Proposes a novel algorithmic framework that leverages causal inference via ANM-MM to extract model parameters and a new data sharing scheme based on similarity measures.
result Demonstrates superior learning speeds on various tasks and effectiveness of diverse action selection.
Improved autoencoder for F0-consistent voice conversion.
problem Non-parallel many-to-many voice conversion with prosodic information leakage.
method Conditional autoencoder with information-constraining bottlenecks.
result Controlled F0 contour and improved speech quality.
We study the interplay between memorization and generalization of overparameterized networks in the extreme case of a single training example and an identity-mapping task. We examine fully-connected and convolutional networks (FCN and CNN), both linear and nonlinear, initialized randomly and then trained to minimize th…
We present the Voice Conversion Challenge 2018, designed as a follow up to the 2016 edition with the aim of providing a common framework for evaluating and comparing different state-of-the-art voice conversion (VC) systems. The objective of the challenge was to perform speaker conversion (i.e. transform the vocal ident…
New research shows non-bottlenecked autoencoders can outperform bottlenecked ones for anomaly detection.
problem The necessity of a bottleneck in autoencoders for anomaly detection.
method Investigated two ways to remove bottlenecks: overparameterising the latent layer and introducing skip connections. Carried out extensive experiments on various AE types and datasets.
result Non-bottlenecked autoencoders can outperform bottlenecked ones, improving anomaly detection performance.
Anti-transfer learning prevents misleading representations for speech tasks.
problem Misleading representations learned from orthogonal tasks in speech processing.
method Penalizes similarity between activations of a network and another trained on an orthogonal task.
result Improves classification accuracy and invariance to the orthogonal task.
It is often stated in papers tackling the task of inferring Bayesian network structures from data that there are these two distinct approaches: (i) Apply conditional independence tests when testing for the presence or otherwise of edges; (ii) Search the model space using a scoring metric. Here I argue that for complete…
New sampling and identity-testing methods for mixtures of distributions that don't satisfy approximate tensorization of entropy.
problem Sampling and identity-testing for mixtures of distributions that don't satisfy approximate tensorization of entropy.
method Fast mixing of Glauber dynamics and efficient identity-testers in the coordinate-conditional sampling access model.
result Efficient identity-testers for mixtures of ATE distributions in the coordinate-conditional sampling access model.
Bayesian Optimization improved for high-dimensional outputs.
problem Optimizing many correlated outcomes efficiently.
method Efficient multi-task Gaussian Process sampling.
result Substantial improvements in sample efficiency.
There are many forms of feature information present in video data. Principle among them are object identity information which is largely static across multiple video frames, and object pose and style information which continuously transforms from frame to frame. Most existing models confound these two types of represen…
A major obstacle to developing artificial intelligence applications capable of true lifelong learning is that artificial neural networks quickly or catastrophically forget previously learned tasks when trained on a new one. Numerous methods for alleviating catastrophic forgetting are currently being proposed, but diffe…
Neural networks struggle with identity relations; DR units improve generalization.
problem Neural networks fail to generalize identity relations.
method Exploring various factors in neural network architecture and learning process, including number of hidden layers, activation function, and data representation.
result DR units improve generalization, leading to almost perfect test accuracy in mid fusion setting.
VRL-SGD reduces communication complexity in non-identical data settings.
problem Training machine learning models with non-identical data distribution.
method VRL-SGD, which eliminates gradient variance dependency and achieves linear speedup with lower communication complexity.
result VRL-SGD reduces communication complexity from $O(T^{ rac{3}{4}} N^{ rac{3}{4}})$ to $O(T^{ rac{1}{2}} N^{ rac{3}{2}})$.
Improves score estimation for noised targets using known clean scores.
problem Poor score estimation at low noise levels in Denoising Score Matching.
method Introduces Target Score Identity and Target Score Matching loss.
result Score estimates are more accurate at low noise levels.
In this paper we study speaker linking (a.k.a.\ partitioning) given constraints of the distribution of speaker identities over speech recordings. Specifically, we show that the intractable partitioning problem becomes tractable when the constraints pre-partition the data in smaller cliques with non-overlapping speakers…
Mimetic initialization improves Transformer training on small datasets.
problem Difficulty in training Transformers on small datasets.
method Initialize self-attention layers to look like pre-trained models.
result Vanilla Transformers trained with mimetic initialization achieve higher accuracy.
This paper discovers new identities linking geodesic and orthogeodesic lengths on hyperbolic surfaces.
problem Understanding relationships between geodesic and orthogeodesic lengths on hyperbolic surfaces.
method Investigates a broad family of identities involving lengths of all closed geodesics and orthogeodesics.
result Introduces new identities that include lengths of all closed geodesics, contrasting with previous identities.
New method estimates animal density using acoustic data, accounting for unknown call identities.
problem Estimating animal density or call density from acoustic data with unknown call identities.
method Monte Carlo Expectation-Maximization (MCEM) method to resolve unknown call identities.
result Estimates are within 15% of expert-constructed estimates and incorporate uncertainty about call identities.
Generative adversarial network synthesizes sketches into realistic images.
problem Improving the quality and realism of facial sketches.
method Hybrid GAN with quality guided encoder and identity preserving network.
result Synthesized images are more realistic and maintain identity.
Establishes a correspondence between two mathematical identities.
problem None explicitly stated; focuses on identity correspondence.
method Establishes correspondence between Pestov and Weitzenböck identities.
result Established correspondence between Pestov and Weitzenböck identities.
Gradient-based meta-learning methods leverage gradient descent to learn the commonalities among various tasks. While previous such methods have been successful in meta-learning tasks, they resort to simple gradient descent during meta-testing. Our primary contribution is the {\em MT-net}, which enables the meta-learner…
Training-free source selection for LLM families with shared vocabularies
problem Source selection for LLM families with shared vocabularies
method Fisher alignment at vocabulary scale
result Fisher alignment is a cosine between kernel mean embeddings in the joint activation-error space
Understanding cell identity is an important task in many biomedical areas. Expression patterns of specific marker genes have been used to characterize some limited cell types, but exclusive markers are not available for many cell types. A second approach is to use machine learning to discriminate cell types based on th…
Proposes using entity embedding vectors to improve Gaussian Process models for knowledge transfer across cell lines.
problem Lack of reuse of experimental data for predicting novel processes.
method Hybrid Gaussian Process models with entity embedding vectors to represent product identity.
result Improved performance in predicting novel processes compared to traditional methods.
Quandle homology was defined from rack homology as the quotient by a subcomplex corresponding to the idempotency, for invariance under the type I Reidemeister move. Similar subcomplexes have been considered for various identities of racks and moves on diagrams. We observe common aspects of these identities and subcompl…
Many-to-Many VTN improves voice conversion across multiple speakers.
problem Voice conversion across multiple speakers.
method Sequence-to-sequence learning framework with many-to-many VTN architecture.
result Improved sound quality and speaker similarity compared to baseline methods.
A new optimizer, Grad-Avg, converges faster than SGD and improves classification task performance.
problem Optimizing error functions in regression and classification tasks.
method Grad-Avg optimizer based on gradient averaging, with mathematical convergence proof and parameter scaling for classification.
result Grad-Avg converges faster than other optimizers for classification tasks on benchmark datasets.
The paper derives curvature identities for 5D and 6D Einstein manifolds.
problem Deriving curvature identities for specific dimensions of Einstein manifolds.
method Using Patterson's curvature identities and the Chern-Gauss-Bonnet Theorem, the paper provides explicit formulae for 5D and 6D Einstein manifolds.
result The curvature identities for 5D and 6D Einstein manifolds are confirmed to be consistent with previous work.
Global Pestov identity proved on frame bundle and related fibrations.
problem Global Pestov identity on frame bundles and fibrations.
method Global Pestov identity on frame bundles and fibrations.
result Global Pestov identity on frame bundles and fibrations.
Proves Bochner's identity on graphs using a new auxiliary graph.
problem Extending Bochner's identity to graph theory.
method Introduces a complete tangent graph to prove the identity.
result Validates Bochner's identity on graphs.
Doodles link to commutator identities in a 2-sphere.
problem Understanding commutator identities in free groups via doodles.
method Analyzing doodles with proper noose systems and establishing bijections.
result A bijection between doodles and commutator identities.
The paper studies harmonic identity maps on Riemannian manifolds.
problem Understanding harmonicity of identity maps on Riemannian manifolds.
method Constructing new examples and defining a symmetric tensor field.
result New examples of identity harmonic maps are constructed.
CLOPS improves deep learning for continuous physiological data.
problem Deep learning struggles with streaming, multi-sensor clinical data.
method CLOPS is a replay-based continual learning strategy.
result CLOPS outperforms state-of-the-art methods in continual learning scenarios.
The importance of Einstein's geometrization philosophy, as an alternative to the least action principle, in constructing general relativity (GR), is illuminated. The role of differential identities in this philosophy is clarified. The use of Bianchi identity to write the field equations of GR is shown. Another similar …
We use computer algebra to demonstrate the existence of a multilinear polynomial identity of degree 8 satisfied by the bilinear operation in every Lie-Yamaguti algebra. This identity is a consequence of the defining identities for Lie-Yamaguti algebras, but is not a consequence of anticommutativity. We give an explicit…