HCRNN uses hierarchical contexts to improve recommendation models.
problem Challenges in modeling user interest transitions and drifts in recommendation systems.
method Introduces HCRNN with three hierarchical contexts (global, local, temporary) and a hierarchical context-based gate structure.
result HCRNN outperformed other models in sequential recommendation tasks.
A new framework generates large hierarchical search spaces for neural architectures.
problem Discovering neural architectures from simple blocks is hard.
method Context-free grammars for a unified, scalable search space.
result Efficiently searches over complete architectures, outperforming existing methods.
Diffusion models learn hierarchical composition rules from data.
problem How many samples do generative models need to learn hierarchical composition rules?
method Theoretical and empirical investigation of diffusion models on probabilistic context-free grammars.
result Diffusion models learn hierarchical composition rules with sample complexity scaling polynomially with context size.
We present a new recurrent neural network topology to enhance state-of-the-art machine learning systems by incorporating a broader context. Our approach overcomes recent limitations with extended narratives through a multi-layered computational approach to generate an abstract context representation. Therefore, the dev…
SCHA-VAE generates novel data from limited examples using hierarchical context aggregation.
problem Generating data from a novel distribution with limited examples.
method Hierarchical context aggregation with attention-based point to set-level aggregation.
result Hierarchical approach better captures intrinsic variability in small data.
Interactions such as double negation in sentences and scene interactions in images are common forms of complex dependencies captured by state-of-the-art machine learning models. We propose Mahé, a novel approach to provide Model-agnostic hierarchical éxplanations of how powerful machine learning models, such as deep ne…
Free Random Projection enhances reinforcement learning by naturally incorporating hierarchical structure.
problem Improving reinforcement learning algorithms for better generalization and adaptability.
method Introduces Free Random Projection, a method that uses free probability theory to create random orthogonal matrices encoding hierarchical structure.
result Empirically shows consistent improvement in generalization over standard methods on multi-environment benchmarks.
This study compares hierarchical and non-hierarchical models for open-domain multi-turn dialog generation.
problem Which kind of models (hierarchical or non-hierarchical) is better for open-domain multi-turn dialog generation?
method Systematically compared nearly all representative hierarchical and non-hierarchical models over the same experimental settings.
result Nearly all hierarchical models are worse than non-hierarchical models in open-domain multi-turn dialog generation, except for HRAN.
Unified theory for neural scaling laws in hierarchically compositional data.
problem Understanding neural scaling laws in hierarchically compositional data.
method Probabilistic context-free grammars and power-law distributed production rules.
result Unified learning curve behavior for classification and next-token prediction tasks.
Transformers prefer simpler explanations in hierarchical tasks.
problem Navigating tasks with varying complexity levels.
method Well-controlled testbeds based on Markov chains and linear regression.
result Transformers favor the least complex sufficient explanation when presented with simpler data.
Log-linear models are the popular workhorses of analyzing contingency tables. A log-linear parameterization of an interaction model can be more expressive than a direct parameterization based on probabilities, leading to a powerful way of defining restrictions derived from marginal, conditional and context-specific ind…
Proposes learning a hierarchical prior in VAEs to avoid over-regularization.
problem Over-regularization in VAEs with standard normal priors.
method Formulates as a constrained optimisation problem, introduces graph-based interpolation.
result Learned latent representation reflects data manifold topology and properties.
Many real-world optimization problems require significant resources for objective function evaluations. This is a challenge to evolutionary algorithms, as it limits the number of available evaluations. One solution are surrogate models, which replace the expensive objective. A particular issue in this context are hiera…
Hierarchical beta process has found interesting applications in recent years. In this paper we present a modified hierarchical beta process prior with applications to hierarchical modeling of multiple data sources. The novel use of the prior over a hierarchical factor model allows factors to be shared across different …
Bayesian framework uses context trees for efficient time series modeling.
problem Building flexible and interpretable mixture models for real-valued time series.
method Hierarchical Bayesian modelling with context trees and autoregressive models.
result The methods outperform state-of-the-art techniques on simulated and real-world experiments.
Model compresses event-like contexts using gated surprise signals.
problem Perceiving a dynamic world as organized events.
method Hierarchical, surprise-gated recurrent neural network architecture.
result Achieves best performance on multiple event processing tasks.
A simple guide to understanding hierarchical causality in complex systems.
problem Understanding hierarchical causality in complex systems.
method Formalizing hierarchical causality in terms of actors and agents, with three key structures.
result The system requires three additional structures: causation classes, aggregation operators, and discrete event-time maps.
A common problem in Machine Learning and statistics consists in detecting whether the current sample in a stream of data belongs to the same distribution as previous ones, is an isolated outlier or inaugurates a new distribution of data. We present a hierarchical Bayesian algorithm that aims at learning a time-specific…
Proves deep networks can learn hierarchical structures efficiently.
problem Understanding how deep networks learn hierarchical structures in data.
method Random Hierarchy Models, gradient-based methods, layerwise training.
result Proves deep networks can efficiently learn hierarchical structures.
Proposes a new RNN for language generation capturing long-range dependencies.
problem Capturing long-range word dependencies and sentence order in text corpora.
method Recurrent Hierarchical Topic-Guided RNN with dynamic deep topic model.
result Outperforms larger-context RNN-based language models and learns interpretable topics.
New model tackles causal bandits with dependent variables.
problem Understanding reward-maximizing interventions in causal networks with dependent variables.
method Introduces hierarchical causal bandit model with a contextual variable capturing interactions among variables.
result Derives nearly matching regret bounds for binary context in causal bandits with dependent arms.
HAXMLNet tackles extreme multi-label text classification with hierarchical attention.
problem Tagging each text with relevant labels from an extreme-scale label set.
method Proposes a hierarchical structure with multi-label attention for efficient and effective XMTC.
result HAXMLNet achieves competitive performance compared to state-of-the-art methods.
HANET combines LSTM and attention mechanisms for better financial forecasting.
problem Lack of distinct macroeconomic regimes in financial datasets.
method Hierarchical Cross-Attention mechanism integrating long-run macro contexts with high-frequency market dynamics.
result HANET outperforms neural forecasters, especially during turbulent periods.
Proposes HBGNN for better recommendation systems using graph neural networks.
problem Sparse structured data in recommendation systems lacking feature richness.
method Hierarchical BiGraph Neural Network (HBGNN) using bigraph framework.
result Competitive performance compared to current methods.
New language model shows context length impacts generation quality and reasoning ability.
problem Analyzing the impact of context length and reasoning on autoregressive generation.
method Introduced synthetic hierarchical languages, used an exact k-gram ansatz, derived asymptotic predictions, and validated empirically.
result Reasoning models with limited context can generate sequences from the true language, improving exponentially over standard models.
Advances combinatorial complexes for better modeling of hierarchical and set-type relations.
problem Lack of effective modeling for complex hierarchical and set-type relations in high-dimensional data.
method Introduces combinatorial complexes as a bridge between cell complexes and hypergraphs, emphasizing their different types of relations.
result Combining set-type and hierarchical relations in a single model can be advantageous in learning tasks.
Variational inference improves hierarchical imitation learning of control programs.
problem Learning structured control policies from demonstrations.
method Variational inference for discovering hierarchical structure in observation-action traces.
result Variational inference leads to more efficient and generalized control policies.
Tag2Vec learns tag representations in hybrid networks with semantic and hierarchical information.
problem Lack of semantic and hierarchical information in tag networks.
method Tag2Vec model that combines nodes and tags into hybrid networks, using parameterized random walks and hyperbolic Skip-gram model.
result Tag2Vec outperforms other models in learning rich semantic tag representations.
We consider a particular instance of a common problem in recommender systems: using a database of book reviews to inform user-targeted recommendations. In our dataset, books are categorized into genres and sub-genres. To exploit this nested taxonomy, we use a hierarchical model that enables information pooling across a…
Within a supervised classification framework, labeled data are used to learn classifier parameters. Prior to that, it is generally required to perform dimensionality reduction via feature extraction. These preprocessing steps have motivated numerous research works aiming at recovering latent variables in an unsupervise…
Bayesian models use hyperparameters to indirectly assign priors, and this work shows how these priors can be derived from maximum entropy principles.
problem Understanding the assumptions and dependencies in Bayesian hierarchical models.
method Demonstrates how canonical distributions and maximum entropy principles can be used to derive marginal priors in hierarchical models.
result Marginal priors in hierarchical models derived from maximum entropy principles have different constraints compared to the original priors.
We compare various extensions of the Bradley-Terry model and a hierarchical Poisson log-linear model in terms of their performance in predicting the outcome of soccer matches (win, draw, or loss). The parameters of the Bradley-Terry extensions are estimated by maximizing the log-likelihood, or an appropriately penalize…
Hierarchical VAEs detect out-of-distribution data by identifying low-level in-distribution features.
problem Out-of-distribution data often has in-distribution low-level features, leading to misleading likelihood estimates in deep generative models.
method Developed a fast, scalable, unsupervised likelihood-ratio score for out-of-distribution detection based on hierarchical variational autoencoders.
result Achieved state-of-the-art results on out-of-distribution detection across various data and model combinations.
The Dirichlet process and its extension, the Pitman-Yor process, are stochastic processes that take probability distributions as a parameter. These processes can be stacked up to form a hierarchical nonparametric Bayesian model. In this article, we present efficient methods for the use of these processes in this hierar…
New algorithm reduces regret in contextual bandits with many near-boundary contexts.
problem High regret in contextual bandits with many near-boundary contexts.
method Hierarchical nearest neighbour approach, holding out contexts for computation.
result Eradicates high regret in adversarial contextual bandits.
A very brief history of relative valuation in neoclassical finance since 1973 is presented, with attention to core currency issues for emerging economies. Price formation is considered in the context of hierarchical causality, with discussion focussed on identifying mathematical modelling challenges for robust and tran…
While we are usually focused on forecasting future values of time series, it is often valuable to additionally predict their entire probability distributions, e.g. to evaluate risk, Monte Carlo simulations. On example of time series of ≈ 30000 Dow Jones Industrial Averages, there will be presented application o…
Paper proposes a deep learning architecture for generating long stories from images.
problem Maintaining context in long event sequences for visual storytelling.
method Hierarchical deep learning architecture with encoder-decoder networks and natural language descriptions.
result Our method outperforms state-of-the-art techniques on automatic evaluation metrics.
Transformer models perform slower than convolutional networks in learning hierarchical language structures.
problem Understanding how neural networks learn hierarchical language structures.
method Theoretical scaling laws and empirical validation of neural network performance.
result Convolutional networks outperform transformers in learning hierarchical language structures.
Unified tensor model disentangles object appearance factors.
problem Representing hierarchical intrinsic and extrinsic causal factors of object appearance.
method Compositional hierarchical tensor factorization.
result Interpretable object representation robust to occlusion and reduced training data requirements.
This paper improves neural network explanations by quantifying and visualizing semantic compositions.
problem Improving neural network explanations for natural language processing tasks.
method Proposes a formal way to quantify word and phrase importance, introduces SCD and SOC algorithms.
result Our algorithms outperform prior methods in explaining neural network predictions.
Geographic object-based image analysis (GEOBIA) framework has gained increasing interest recently. Following this popular paradigm, we propose a novel multiscale classification approach operating on a hierarchical image representation built from two images at different resolutions. They capture the same scene with diff…
Study shows how leveraging hierarchical similarity graphs improves matrix completion in recommender systems.
problem Improving matrix completion in recommender systems using hierarchical similarity graphs.
method Characterizes the optimal sample complexity using hierarchical stochastic block models and low-rank rating matrices.
result Exploiting hierarchical structure of social graphs significantly reduces the number of observed entries needed for accurate matrix completion.
Multi-Output Dependence (MOD) learning is a generalization of standard classification problems that allows for multiple outputs that are dependent on each other. A primary issue that arises in the context of MOD learning is that for any given input pattern there can be multiple correct output patterns. This changes the…
New framework estimates staged tree models using hierarchical clustering on the probability simplex.
problem Estimating staged tree models with context-specific dependencies.
method Hierarchical clustering on the probability simplex, using simplex-based divergences and linkage methods.
result Total Variation divergence with Ward.D2 linkage produces staged trees with better model fit, structure recovery, and computational efficiency.
Improved patient classification using preprocessed clinical notes.
problem Classifying patient discharges with clinical notes, which are often messy and hard to process.
method Pretrained hierarchical recurrent neural network model that parses clinical notes without preprocessing.
result The model improves discharge diagnosis classification performance on MIMIC-III dataset.
Learning an optimal policy from a multi-modal reward function is a challenging problem in reinforcement learning (RL). Hierarchical RL (HRL) tackles this problem by learning a hierarchical policy, where multiple option policies are in charge of different strategies corresponding to modes of a reward function and a gati…
We optimize ultrametric fitting for hierarchical clustering.
problem Hierarchical clustering methods lack a clear optimization framework.
method We reformulate ultrametric fitting as a constrained optimization problem over continuous ultrametrics, solving it with gradient descent.
result Gradient descent efficiently solves the reformulated optimization problem, leading to good performance compared to state-of-the-art methods.