Two new Hie-TAN and Hie-TAN-Lite algorithms improve TAN for hierarchical feature spaces.
problem Learning dependencies in hierarchical feature spaces.
method Exploits hierarchical parent-child relationships as constraints to learn a dependency tree.
result Hie-TAN-Lite outperforms Hie-TAN and other methods in predictive accuracy.
There has been a surge in the number of large and flat data sets - data sets containing a large number of features and a relatively small number of observations - due to the growing ability to collect and store information in medical research and other fields. Hierarchical clustering is a widely used clustering tool. I…
Deep neural networks have been shown to be very successful at learning feature hierarchies in supervised learning tasks. Generative models, on the other hand, have benefited less from hierarchical models with multiple layers of latent variables. In this paper, we prove that hierarchical latent variable models do not ta…
Method learns hierarchical representations of samples and features simultaneously.
problem Hierarchical structures in samples and features not considered by existing methods.
method Jointly learns hierarchical representations via Tree-Wasserstein Distance alternating between samples and features.
result Method improves performance in link prediction and node classification tasks.
Three-layer networks learn complex hierarchical polynomials of multiple nonlinear features.
problem Understanding how neural networks learn hierarchical features of multiple nonlinear inputs.
method Examine a broad class of functions using three-layer neural networks, showing complete recovery and efficient learning.
result Three-layer neural networks trained via gradient descent can learn hierarchical polynomials of multiple nonlinear features efficiently.
New hierarchical VQ-VAE scheme improves image compression quality and features at low bitrates.
problem Low bitrate image compression maintaining quality and features.
method Hierarchical VQ-VAE with stochastic quantization and Markovian latent variables.
result High perceptual quality and semantic features at low bitrates.
Classification with Costly Features (CwCF) is a classification problem that includes the cost of features in the optimization criteria. Individually for each sample, its features are sequentially acquired to maximize accuracy while minimizing the acquired features' cost. However, existing approaches can only process da…
DHRL learns interpretable features from visual data.
problem Limited use of deep learning in basic research for interpretable features.
method Generative model chaining, ladder network architecture, latent space regularization.
result DHRL generates disentangled hierarchical features from small datasets.
A new method for linear regression using feature graphs and hierarchical shrinkage.
problem Estimating robust parameters for linear regression models.
method Hierarchical Feature Regression (HFR) estimator that constructs a supervised feature graph to shrink parameters towards group targets.
result Demonstrates good predictive accuracy and versatility compared to other regularization techniques.
A hierarchical model shows how scaling laws emerge from sequential feature recovery.
problem Emergence of scaling laws from feature learning in multi-layer networks.
method Layer-wise spectral algorithm adapted to compositional structure, sequential feature detection.
result Sequential detection of latent features, leading to explicit power-law decay of prediction error.
Deep learning explained through spectral filtering of hierarchical features.
problem Understanding how deep neural networks learn useful representations from data.
method Neural Low-Degree Filtering (Neural LoFi) as a stylized limit of gradient-based training.
result Predicts how representations are selected layer by layer and explains emergence of concepts.
Proposes HBGNN for better recommendation systems using graph neural networks.
problem Sparse structured data in recommendation systems lacking feature richness.
method Hierarchical BiGraph Neural Network (HBGNN) using bigraph framework.
result Competitive performance compared to current methods.
A new convolutional spectral kernel network learns hierarchical and local features.
problem Lack of deep learning in non-stationary spectral kernels.
method Introduces convolutional filters and deep architectures into non-stationary spectral kernels, derives generalization error bounds, and introduces regularizers.
result Validated the effectiveness of the convolutional spectral kernel network on real-world datasets.
Hierarchical VAEs detect out-of-distribution data by identifying low-level in-distribution features.
problem Out-of-distribution data often has in-distribution low-level features, leading to misleading likelihood estimates in deep generative models.
method Developed a fast, scalable, unsupervised likelihood-ratio score for out-of-distribution detection based on hierarchical variational autoencoders.
result Achieved state-of-the-art results on out-of-distribution detection across various data and model combinations.
Paper develops a framework for generating coherent image captions using visual features and hierarchical topics.
problem Generating semantically coherent paragraphs to describe image content.
method Plug-and-play hierarchical-topic-guided image paragraph generation framework integrating visual extractor and deep topic model.
result Proposed models can distill interpretable multi-layer semantic topics and generate diverse and coherent captions.
Proves deep networks can learn hierarchical structures efficiently.
problem Understanding how deep networks learn hierarchical structures in data.
method Random Hierarchy Models, gradient-based methods, layerwise training.
result Proves deep networks can efficiently learn hierarchical structures.
Unified framework for modeling hierarchical spaces in design problems.
problem Challenges in modeling hierarchical, conditional, heterogeneous, or tree-structured domains.
method Unified framework combining feature modeling and graph theory, introducing meta and partially-decreed variables.
result Demonstrated effectiveness on complex system design problems, including neural networks and green-aircraft.
Paper proposes a GPU-based system for training massive deep learning models in ads systems.
problem Training massive deep learning models with terabyte-scale parameters in ads systems.
method Hierarchical GPU parameter server with 3-layer storage (GPU High-Bandwidth Memory, CPU main memory, SSD).
result 4-node hierarchical GPU parameter server trains a model 2X faster than a 150-node in-memory system.
We propose a novel hierarchical generative model with a simple Markovian structure and a corresponding inference model. Both the generative and inference model are trained using the adversarial learning paradigm. We demonstrate that the hierarchical structure supports the learning of progressively more abstract represe…
Deep MF extracts hierarchical features from large data sets.
problem Mining complex, interleaved features in large data sets.
method Deep matrix factorization models and algorithms.
result Deep MF achieves outstanding performance on unsupervised tasks.
Neural NMF discovers hierarchical topics in multilayer data.
problem Detecting latent hierarchical structure in multilayer data.
method Recursive application of nonnegative matrix factorization (NMF) in layers with backpropagation optimization.
result Neural NMF outperforms other hierarchical NMF methods in synthetic and real-world datasets.
Proposes a hierarchical clustering method for positive and negative dissimilarities.
problem Clustering dissimilarities, especially positive and negative.
method Hierarchical correlation clustering followed by tree preserving embedding.
result Performance on various datasets.
Deep learning is also known as hierarchical learning, where the learner _learns_ to represent a complicated target function by decomposing it into a sequence of simpler functions to reduce sample and time complexity. This paper formally analyzes how multi-layer neural networks can perform such hierarchical learning _ef…
Bayesian model estimates feature values of premium products.
problem Understanding the monetary value of product features.
method Bayesian Hierarchical Logit Model
result Successfully recovers true feature valuations from data.
Boxhead dataset tests autoencoder disentanglement in hierarchical data.
problem Evaluate disentanglement in hierarchical data.
method Introduced Boxhead dataset with hierarchically structured factors, evaluated autoencoder models.
result Hierarchical models outperform single-layer VAEs in disentangling factors.
Architectures for sparse hierarchical representation learning have recently been proposed for graph-structured data, but so far assume the absence of edge features in the graph. We close this gap and propose a method to pool graphs with edge features, inspired by the hierarchical nature of chemistry. In particular, we …
Contrastive learning properties studied, including feature suppression and hierarchical learning.
problem Feature suppression and hierarchical learning in contrastive learning.
method Generalized contrastive loss, instance-based contrastive learning, explicit and controllable competing features.
result Contrastive learning can suppress and prevent the learning of competing features.
In classification problems, especially those that categorize data into a large number of classes, the classes often naturally follow a hierarchical structure. That is, some classes are likely to share similar structures and features. Those characteristics can be captured by considering a hierarchical relationship among…
Diffusion models learn hierarchical composition rules from data.
problem How many samples do generative models need to learn hierarchical composition rules?
method Theoretical and empirical investigation of diffusion models on probabilistic context-free grammars.
result Diffusion models learn hierarchical composition rules with sample complexity scaling polynomially with context size.
Three-layer neural networks learn hierarchical polynomial functions efficiently.
problem Learning hierarchical polynomial functions with three-layer neural networks.
method Layerwise gradient descent on square loss, focusing on feature learning.
result Achieves optimal sample complexity for learning hierarchical polynomials.
Deep networks learn hierarchical data by invariant representations.
problem How many examples are needed for deep networks to learn hierarchical data?
method Random Hierarchy Model: synthetic tasks inspired by language and images hierarchy.
result Deep networks learn by invariant representations and require a detectable number of correlations between low-level features and classes.
Recent progress on deep learning has made it possible to automatically transform the screenshot of Graphic User Interface (GUI) into code by using the encoder-decoder framework. While the commonly adopted image encoder (e.g., CNN network), might be capable of extracting image features to the desired level, interpreting…
HiGraphDTI learns drug and target representations from molecular graphs to predict DTIs.
problem Inaccurate drug-target interaction prediction due to insufficient chemical information extraction.
method Hierarchical graph representation learning to extract chemical information from atoms, motifs, and molecules.
result HiGraphDTI outperforms state-of-the-art methods in DTI prediction and interaction interpretation.
Deep learning models based on CNNs are predominantly used in image classification tasks. Such approaches, assuming independence of object categories, normally use a CNN as a feature learner and apply a flat classifier on top of it. Object classes in many settings have hierarchical relations, and classifiers exploiting …
Deep networks, composed of multiple layers of hierarchical distributed representations, tend to learn low-level features in initial layers and transition to high-level features towards final layers. Paradigms such as transfer learning, multi-task learning, and continual learning leverage this notion of generic hierarch…
Diffusion models reveal a phase transition in reconstructing high-level features.
problem Understanding the hierarchical structure of natural data.
method Study of hierarchical generative models of data using diffusion models.
result The backward diffusion process shows a phase transition at a threshold time, where high-level features suddenly drop in reconstructibility.
We present an approach to model-based hierarchical clustering by formulating an objective function based on a Bayesian analysis. This model organizes the data into a cluster hierarchy while specifying a complex feature-set partitioning that is a key component of our model. Features can have either a unique distribution…
Proposes a graph pooling method leveraging node proximity for hierarchical graph representation learning.
problem Efficiently exploiting the geometry of graph data for hierarchical representation learning.
method Combines node proximity with kernel representation of topology and node features for adaptive node signal similarities evaluation.
result Achieves state-of-the-art performance on graph classification benchmark datasets.
DHGAK aligns substructures for better graph kernel performance.
problem Limited performance of traditional graph kernels due to missing substructure similarities.
method Hierarchically aligns relational substructures in deep embedding space, assigning same feature maps in RKHS.
result DHGAK outperforms state-of-the-art graph kernels on various benchmarks.
We introduce the hierarchical compositional network (HCN), a directed generative model able to discover and disentangle, without supervision, the building blocks of a set of binary images. The building blocks are binary features defined hierarchically as a composition of some of the features in the layer immediately be…
HH-VAEM improves imputation and acquisition of missing data using hierarchical models and Hamiltonian Monte Carlo.
problem Imputation and acquisition of missing heterogeneous data.
method Hierarchical VAE model with Hamiltonian Monte Carlo and automatic hyper-parameter tuning.
result HH-VAEM outperforms existing methods in imputation and supervised learning tasks.
Deep networks learn hierarchical functions more efficiently than shallow ones.
problem Understanding the advantage of deep neural networks over shallow models.
method Analytical study of learning dynamics and generalization performance of deep networks compared to shallow ones.
result Deep networks reduce effective dimensionality, enabling learning with fewer samples.
The staircase property aids deep learning by guiding hierarchical feature learning.
problem Understanding how hierarchical structure influences deep learning performance.
method Defined and proved the staircase property for Boolean hypercube functions, and showed its learnability by layerwise stochastic coordinate descent.
result Staircase functions can be learned in polynomial time using layerwise stochastic coordinate descent on regular neural networks.
CoHiRF extends clustering methods to handle high-dimensional data efficiently.
problem Scalability limits of existing clustering methods.
method Hierarchical consensus framework operating on label assignments.
result Improves robustness and scalability to high-dimensional noise.
Sum-Product Networks (SPNs) are a class of expressive yet tractable hierarchical graphical models. LearnSPN is a structure learning algorithm for SPNs that uses hierarchical co-clustering to simultaneously identifying similar entities and similar features. The original LearnSPN algorithm assumes that all the variables …
Nowadays, hyperspectral image classification widely copes with spatial information to improve accuracy. One of the most popular way to integrate such information is to extract hierarchical features from a multiscale segmentation. In the classification context, the extracted features are commonly concatenated into a lon…
Slow feature analysis (SFA) is an unsupervised-learning algorithm that extracts slowly varying features from a multi-dimensional time series. A supervised extension to SFA for classification and regression is graph-based SFA (GSFA). GSFA is based on the preservation of similarities, which are specified by a graph struc…
Paper proposes a new method for selecting the best hierarchical forecasting approach.
problem Selecting the best method for reconciling base forecasts in hierarchical time series.
method Conditional hierarchical forecasting using machine learning and time series features.
result Conditional hierarchical forecasting leads to significantly more accurate forecasts, especially at lower levels.