HCL learns shared and modality-specific latent representations for multimodal data.
problem Binary shared-private decomposition inadequately represents shared information across subsets of modalities.
method Hierarchical Contrastive Learning framework combining latent-variable formulation, structural sparsity, and contrastive objective.
result HCL accurately recovers hierarchical structure and improves predictive performance on multimodal data.
Contrastive learning properties studied, including feature suppression and hierarchical learning.
problem Feature suppression and hierarchical learning in contrastive learning.
method Generalized contrastive loss, instance-based contrastive learning, explicit and controllable competing features.
result Contrastive learning can suppress and prevent the learning of competing features.
Framework explains how dual deep networks learn features from unlabeled data.
problem Understanding self-supervised learning with dual deep networks.
method Theoretical framework and hierarchical latent tree model.
result Deep ReLU networks learn latent variables through contrastive SSL.
Paper develops a theory explaining contrastive pre-training for multimodal AI.
problem Limited theoretical understanding of contrastive pre-training for multi-modal AI.
method Introduces approximate sufficient statistics and Joint Generative Hierarchical Model.
result Near-minimizers of contrastive loss are approximately sufficient, enabling diverse downstream tasks.
Hyperbolic space outperforms Euclidean in learning hierarchical data.
problem Learning hierarchical data in Euclidean space requires exponentially many samples.
method Established geometric obstruction in Euclidean space and showed hyperbolic space's advantage.
result Hyperbolic space enables learning with O(mRlogm) samples, matching information-theoretic optimum. A novel framework for adaptive multi-agent communication in reinforcement learning.
problem Manual specification of communication structures in multi-agent reinforcement learning.
method Learning Structured Communication (LSC) framework using hierarchical graph neural networks.
result Adaptive hierarchical formations and efficient message propagation among agents.
Improved VAEs by training a contrastive prior to match posterior.
problem Prior hole problem in VAEs, leading to poor image generation.
method Introduced a contrastive energy-based prior and trained it using noise contrastive estimation.
result Significant improvement in VAE generative performance on various datasets.
Unified theory for neural scaling laws in hierarchically compositional data.
problem Understanding neural scaling laws in hierarchically compositional data.
method Probabilistic context-free grammars and power-law distributed production rules.
result Unified learning curve behavior for classification and next-token prediction tasks.
Improves few-shot learning for hierarchical data using hyperbolic space.
problem Few-shot class-incremental learning for hierarchical data.
method Contrastive learning in hyperbolic space, Poincaré ball model, hyperbolic contrastive loss, maximum entropy distribution.
result Effective improvement of coarse and fine class accuracies in few-shot conditions.
Unified framework for semi-supervised learning reduces annotation needs.
problem Sparse annotations and large amounts of unlabeled data in computational pathology.
method S5CL integrates fully-supervised, self-supervised, and semi-supervised learning through hierarchical contrastive losses.
result S5CL improves accuracy and F1-score in histopathological datasets with sparse labels.
Infinite hierarchical contrastive clustering identifies personal environments linked to health outcomes.
problem Identifying meaningful relationships between environmental features and health outcomes on an individual level.
method Contrastive clustering framework with stick-breaking prior and participant-specific prediction loss.
result Model effectively identifies distinct personal environments and groups them into meaningful types linked to health outcomes.
Extends linear representation hypothesis to categorical and hierarchical concepts in LLMs.
problem Representing concepts without natural contrasts in large language models.
method Formalizes linear representation hypothesis for categorical and hierarchical concepts, proving relationships between concept hierarchy and representation geometry.
result Validated theoretical results on large language models, estimating representations for 900+ concepts.
Learning-to-learn or meta-learning leverages data-driven inductive bias to increase the efficiency of learning on a novel task. This approach encounters difficulty when transfer is not advantageous, for instance, when tasks are considerably dissimilar or change over time. We use the connection between gradient-based me…
Enhances thematic investing with stock embeddings from textual data.
problem Challenges in constructing thematic portfolios due to overlapping sector boundaries and evolving market dynamics.
method Introduces THEME, a framework that fine-tunes embeddings using hierarchical contrastive learning, aligning themes and stocks using their hierarchical relationship and incorporating stock returns.
result Theme-aligned portfolios demonstrate compelling performance, significantly outperforming large language models in thematic asset retrieval.
A clustering algorithm for natural hierarchical clusters with near-linear time complexity.
problem Hierarchical clustering with near-linear time complexity.
method Nearest neighbor based clustering algorithm that defines clusters naturally.
result Near-linear time and space complexity for certain datasets.
The staircase property aids deep learning by guiding hierarchical feature learning.
problem Understanding how hierarchical structure influences deep learning performance.
method Defined and proved the staircase property for Boolean hypercube functions, and showed its learnability by layerwise stochastic coordinate descent.
result Staircase functions can be learned in polynomial time using layerwise stochastic coordinate descent on regular neural networks.
CNPs improve function approximation by contrastive learning.
problem Learning from non-i.i.d function instantiations in high-dimensional, noisy spaces.
method CNPs with TCL and FCL contrastive branches for better function approximation.
result CNPs outperform other variants in function distribution reconstruction and parameter identification.
Classifies 3-manifold groups with equivariant hierarchically hyperbolic structures.
problem Classifying 3-manifold groups with equivariant hierarchically hyperbolic structures.
method Construction of suitable quasimorphisms on Seifert pieces to construct actions on quasi-lines.
result 3-manifold groups admit equivariant hierarchically hyperbolic structures.
Deep learning is also known as hierarchical learning, where the learner _learns_ to represent a complicated target function by decomposing it into a sequence of simpler functions to reduce sample and time complexity. This paper formally analyzes how multi-layer neural networks can perform such hierarchical learning _ef…
JojoSCL improves scRNA-seq clustering by reducing intra-cluster dispersion.
problem High dimensionality and sparsity of scRNA-seq data challenge clustering models.
method Integrates shrinkage estimator and contrastive learning for improved clustering.
result JojoSCL outperforms existing methods on ten scRNA-seq datasets.
INVERT connects neural representations to human-understandable concepts.
problem Lack of understanding and statistical significance in existing explainability methods.
method Inverse Recognition (INVERT) approach that connects learned representations to human-understandable concepts.
result INVERT provides interpretable metrics and statistical significance for representation alignment.
Framework detects covert financial market manipulation using LOB representations.
problem Detecting covert financial market manipulation (spoofing) from complex anomaly patterns in multilevel prices.
method Cascaded contrastive representation learning of LOB data.
result Transformer-based architectures achieve state-of-the-art results in detection performance.
A new benchmark task for evaluating policy learning in complex, high-dimensional action spaces.
problem Lack of a commonly accepted benchmark for evaluating policy learning in hierarchical tasks with high-dimensional action spaces.
method Proposed DinerDash Gym benchmark and Decomposed Policy Graph Modelling (DPGM) algorithm.
result DPGM achieves significant improvement over baselines and effectively injects domain knowledge.
The Baire metric induces an ultrametric on a dataset and is of linear computational complexity, contrasted with the standard quadratic time agglomerative hierarchical clustering algorithm. In this work we evaluate empirically this new approach to hierarchical clustering. We compare hierarchical clustering based on the …
New MMM captures hierarchical marketing effects and sign restrictions.
problem Measuring effectiveness of marketing activities with hierarchical structure and sign constraints.
method Proposes a constrained maximum likelihood approach using Hamiltonian Monte Carlo algorithm.
result Demonstrates superior performance on real datasets compared to multi-stage methods.
Local-HDP learns independent topics for each 3D object category in real-time.
problem Learning independent topics for each 3D object category in real-time.
method Local-Hierarchical Dirichlet Process (Local-HDP) with online variational inference.
result Local-HDP outperforms other approaches in accuracy, scalability, and memory efficiency.
This work tackles long-term visual planning by goal-conditioned hierarchical predictors.
problem Current learning approaches fail on long-horizon tasks due to lack of goal information and coarse-to-fine planning.
method Formulate goal-conditioned predictors (GCPs) and hierarchical models to predict trajectories between observations.
result GCPs enable effective long-term planning with much longer horizons than before.
Tail-GNNs improve protein function prediction using relational reinforcement.
problem Predicting hierarchical protein functions from sequence data.
method Combining Tail-GNNs with dilated convolutional networks for multi-task learning.
result Significant improvement in F_1 score for protein function prediction.
Study compares local and global models for hierarchical forecasting accuracy.
problem Challenges in hierarchical time series forecasting, especially in accuracy and information utilisation.
method Developed and evaluated local and global forecasting models (GFMs) to exploit cross-series and cross-hierarchies information.
result Global Forecasting Models (GFMs) outperform local models in hierarchical forecasting accuracy and computational efficiency.
Neural networks benefit from intermediate representations, reducing sample complexity.
problem Understanding how neural networks leverage intermediate representations for hierarchical learning.
method Fixed, randomly initialized neural network as a representation function, compared with raw inputs and other trainable networks.
result Neural representations can achieve improved sample complexities compared to raw inputs, especially for low-rank polynomials.
Exact hierarchical clustering algorithms for data analysis.
problem Finding meaningful structures in data like phylogenetic trees.
method Novel dynamic-programming algorithms based on a trellis data structure.
result Exact computation of partition function, maximum likelihood hierarchy, and marginal probabilities.
Bottom-up algorithms outperform top-down in hierarchical community detection at intermediate levels.
problem Finding the optimal hierarchical community structure in networks.
method A bottom-up algorithm for hierarchical clustering of networks.
result Bottom-up algorithms achieve the information-theoretic threshold for exact recovery at intermediate levels of the hierarchy.
We introduce a simple analysis of the structural complexity of infinite-memory processes built from random samples of stationary, ergodic finite-memory component processes. Such processes are familiar from the well known multi-arm Bandit problem. We contrast our analysis with computation-theoretic and statistical infer…
We address the problem of learning hierarchical deep neural network policies for reinforcement learning. In contrast to methods that explicitly restrict or cripple lower layers of a hierarchy to force them to use higher-level modulating signals, each layer in our framework is trained to directly solve the task, but acq…
Variable selection for models including interactions between explanatory variables often needs to obey certain hierarchical constraints. The weak or strong structural hierarchy requires that the existence of an interaction term implies at least one or both associated main effects to be present in the model. Lately, thi…
In this work we present a clustering technique called \textit{multi-level conformal clustering (MLCC)}. The technique is hierarchical in nature because it can be performed at multiple significance levels which yields greater insight into the data than performing it at just one level. We describe the theoretical underpi…
Proposes a new approach to time series representation learning by embedding patches independently.
problem Capturing dependencies between time series patches is not optimal for representation learning.
method 1) Patch reconstruction task, 2) Patch-wise MLP, 3) Complementary contrastive learning.
result Improves time series forecasting and classification performance compared to state-of-the-art models.
In regression tasks the distribution of the data is often too complex to be fitted by a single model. In contrast, partition-based models are developed where data is divided and fitted by local models. These models partition the input space and do not leverage the input-output dependency of multimodal-distributed data,…
Curious hierarchical reinforcement learning improves learning performance.
problem Combining hierarchical abstraction and curiosity-driven exploration in reinforcement learning.
method Developed a method that combines hierarchical reinforcement learning with curiosity.
result Curiosity can more than double learning performance and success rates.
Improved 3D generative models for drug design reduce bias and enhance data efficiency.
problem Data sparsity and bias in 3D molecular design models.
method Multi-level contrastive learning protocol for bias control and data efficiency.
result Hierarchical generative models that are topologically unbiased and explainable.
Proves EM algorithm guarantees for hierarchical imitation learning.
problem Learning hierarchical policies from expert demonstrations.
method Converts hierarchical imitation learning to latent variable model inference, analyzes EM algorithm.
result Proves EM algorithm converges to true parameter under regular conditions.
Proves deep networks can learn hierarchical structures efficiently.
problem Understanding how deep networks learn hierarchical structures in data.
method Random Hierarchy Models, gradient-based methods, layerwise training.
result Proves deep networks can efficiently learn hierarchical structures.
Develops hierarchical reinforcement learning value function approximators.
problem Estimating long-term returns in reinforcement learning with multiple goals.
method Introduces hierarchical universal value function approximators (H-UVFAs) using the options framework.
result Demonstrates generalization and improved performance of H-UVFAs over UVFAs.
Novel online algorithm for hierarchical imitation learning.
problem Scalability issue in reinforcement learning and options discovery.
method Online Baum-Welch algorithm for hierarchical imitation learning.
result The online algorithm outperforms the batch version in both discrete and continuous environments.
Proposes a hierarchical clustering method for positive and negative dissimilarities.
problem Clustering dissimilarities, especially positive and negative.
method Hierarchical correlation clustering followed by tree preserving embedding.
result Performance on various datasets.
Unsupervised method learns hierarchical graph representations without labels.
problem Lack of hierarchical graph representations and need for labeled data in GNNs.
method Maximizes mutual information between local and global graph representations.
result Comparable performance to supervised methods on graph classification benchmarks.
Deep neural networks have been shown to be very successful at learning feature hierarchies in supervised learning tasks. Generative models, on the other hand, have benefited less from hierarchical models with multiple layers of latent variables. In this paper, we prove that hierarchical latent variable models do not ta…
The paper extends fairness to hierarchical clustering, finding efficient algorithms with minimal loss.
problem Ensuring fairness in hierarchical clustering where data is recursively partitioned.
method Extending fairness to hierarchical clustering, developing simple, efficient algorithms for various objectives.
result Simple, efficient algorithms for fair hierarchical clustering with only a negligible loss in objective.