Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

72144216288 · Jun 202019922001200920172026
48 results for hierarchical regularization

Posterior regularization enhances Bayesian hierarchical mixture clustering by improving node separation.

problem High nodal variance in BHMC trees, leading to weak separation between nodes at higher levels.
method Employing Posterior Regularization to impose max-margin constraints on nodes at every level.
result Improves cluster separation in BHMC models, enhancing overall model performance.

The paper explains implicit regularization in hierarchical tensor factorization and deep CNNs.

problem Understanding implicit regularization in complex neural network architectures.
method Theoretical analysis using dynamical systems to overcome challenges in hierarchy.
result Established implicit regularization towards low hierarchical tensor rank, equivalent to locality in CNNs.

Improved prediction of hierarchical time series using structured regularization.

problem Making coherent forecasts for hierarchical time series.
method Structured regularization method for bottom-level time series predictions.
result Superior prediction accuracy and computational efficiency compared to previous methods.

A new method for linear regression using feature graphs and hierarchical shrinkage.

problem Estimating robust parameters for linear regression models.
method Hierarchical Feature Regression (HFR) estimator that constructs a supervised feature graph to shrink parameters towards group targets.
result Demonstrates good predictive accuracy and versatility compared to other regularization techniques.

Diffusion models generalize better with hierarchical data structure and regularization.

problem Understanding generalization in diffusion models with finite data.
method Analyzing diffusion models through data covariance spectra and developing a theoretical framework based on linear neural networks.
result Generalization in diffusion models improves with hierarchical data structure and regularization.

Hyperbolic space outperforms Euclidean in learning hierarchical data.

problem Learning hierarchical data in Euclidean space requires exponentially many samples.
method Established geometric obstruction in Euclidean space and showed hyperbolic space's advantage.
result Hyperbolic space enables learning with O(mRlogm)O(mR \log m) samples, matching information-theoretic optimum.

Dropout is a very effective method in preventing overfitting and has become the go-to regularizer for multi-layer neural networks in recent years. Hierarchical mixture of experts is a hierarchically gated model that defines a soft decision tree where leaves correspond to experts and decision nodes correspond to gating …

2018-12-25abs ↗pdf ↗

Sparse model for noisy datasets using hierarchical regularization.

problem Learning from large noisy datasets with sparse representations.
method Hierarchical learning strategy with projection-based penalty operators.
result Efficient sparse model reconstruction and generalizability on real datasets.

The paper extends NSGPs with L1L^1-regularization for sparsity and solves the resulting R-NSGP regression problem.

problem Sparsity in non-stationary temporal data.
method Developed an ADMM-based method for solving the regularized NSGP regression problem.
result The proposed methods induce sparsity in the parameters of NSGPs.

An important problem in multi-label classification is to capture label patterns or underlying structures that have an impact on such patterns. This paper addresses one such problem, namely how to exploit hierarchical structures over labels. We present a novel method to learn vector representations of a label space give…

2014-12-22abs ↗pdf ↗

Deep neural networks have been shown to be very successful at learning feature hierarchies in supervised learning tasks. Generative models, on the other hand, have benefited less from hierarchical models with multiple layers of latent variables. In this paper, we prove that hierarchical latent variable models do not ta…

2017-02-27abs ↗pdf ↗

Method solves learning problem with hierarchical control objectives.

problem Learning high-dimensional nonlinear functions with model validation accuracy.
method Successive approximation method in functional spaces for hierarchical optimal control.
result Nested algorithm for solving optimal control problem.

A new convolutional spectral kernel network learns hierarchical and local features.

problem Lack of deep learning in non-stationary spectral kernels.
method Introduces convolutional filters and deep architectures into non-stationary spectral kernels, derives generalization error bounds, and introduces regularizers.
result Validated the effectiveness of the convolutional spectral kernel network on real-world datasets.

Variational dropout (VD) is a generalization of Gaussian dropout, which aims at inferring the posterior of network weights based on a log-uniform prior on them to learn these weights as well as dropout rate simultaneously. The log-uniform prior not only interprets the regularization capacity of Gaussian dropout in netw…

2018-11-19abs ↗pdf ↗

HEBAE improves VAEs by adaptively balancing reconstruction and regularization.

problem Posterior collapse in VAEs leading to over-regularization and poor latent encoding.
method Hierarchical Empirical Bayes approach to probabilistic generative models.
result HEBAE generates higher quality samples with better FID scores.

Paper proposes a Renyi entropy-based method for tuning hierarchical topic models.

problem Tuning hierarchical topic models, especially determining the number of topics at each level, is challenging.
method The paper introduces a Renyi entropy-based metric for quality assessment and a practical tuning concept.
result The proposed method can estimate the number of topics for two hierarchical levels in hARTM model.

The paper develops a decision support system for hierarchical text classification of conference proceedings.

problem Classifying documents with a fixed hierarchical structure of topics.
method Developed a weighted hierarchical similarity function to calculate topic relevance, using entropy of words to estimate weights.
result The weighted hierarchical similarity function improves ranking accuracy compared to other methods.

A new method for accurately reconstructing signals without knowing the kernel or signal regularity.

problem Recovering signals from noisy measurements without prior knowledge of the convolution kernel or signal regularity.
method Parametrizing the convolution kernel and prior length-scales, jointly estimated in the inversion procedure.
result Accurate reconstructions of signals with varying regularity and unknown kernel size.

Hierarchical graph clustering is a common technique to reveal the multi-scale structure of complex networks. We propose a novel metric for assessing the quality of a hierarchical clustering. This metric reflects the ability to reconstruct the graph from the dendrogram, which encodes the hierarchy. The optimal represent…

2018-07-13abs ↗pdf ↗

A new framework generates large hierarchical search spaces for neural architectures.

problem Discovering neural architectures from simple blocks is hard.
method Context-free grammars for a unified, scalable search space.
result Efficiently searches over complete architectures, outperforming existing methods.

Paper analyzes consistency of Bayesian and machine learning methods for hierarchical parameter estimation.

problem Learning hierarchical parameters in complex and real-world problems.
method Empirical Bayes and Kernel Flow approaches.
result Consistency results for Matérn-like model on the torus, and comparison of algorithms.

We present Multitask Soft Option Learning(MSOL), a hierarchical multitask framework based on Planning as Inference. MSOL extends the concept of options, using separate variational posteriors for each task, regularized by a shared prior. This ''soft'' version of options avoids several instabilities during training in a …

2019-04-01abs ↗pdf ↗

We propose a new method for learning word representations using hierarchical regularization in sparse coding inspired by the linguistic study of word meanings. We show an efficient learning algorithm based on stochastic proximal methods that is significantly faster than previous approaches, making it possible to perfor…

2014-06-08abs ↗pdf ↗

The staircase property aids deep learning by guiding hierarchical feature learning.

problem Understanding how hierarchical structure influences deep learning performance.
method Defined and proved the staircase property for Boolean hypercube functions, and showed its learnability by layerwise stochastic coordinate descent.
result Staircase functions can be learned in polynomial time using layerwise stochastic coordinate descent on regular neural networks.

Convex clustering is a recent stable alternative to hierarchical clustering. It formulates the recovery of progressively coalescing clusters as a regularized convex problem. While convex clustering was originally designed for handling Euclidean distances between data points, in a growing number of applications, the dat…

2019-11-08abs ↗pdf ↗

DHRL learns interpretable features from visual data.

problem Limited use of deep learning in basic research for interpretable features.
method Generative model chaining, ladder network architecture, latent space regularization.
result DHRL generates disentangled hierarchical features from small datasets.

We propose a new class of convex penalty functions, called \emph{variational Gram functions} (VGFs), that can promote pairwise relations, such as orthogonality, among a set of vectors in a vector space. These functions can serve as regularizers in convex optimization problems arising from hierarchical classification, m…

2015-07-16abs ↗pdf ↗

NVAE improves VAE performance on large image datasets.

problem Improving variational autoencoder performance for large image datasets.
method Deep hierarchical VAE with depth-wise separable convolutions and batch normalization, residual parameterization of Normal distributions, and spectral regularization.
result NVAE achieves state-of-the-art results on MNIST, CIFAR-10, CelebA 64, and CelebA HQ datasets.

New algorithms adaptively compete against complex environments with local regularities.

problem Efficiently competing against complex, locally regular comparator functions in nonparametric settings.
method Locally-adaptive online algorithms using hierarchical εε-nets and tree experts.
result Proved regret bounds scaling with different types of local regularities, delivering better performance for simple profiles.

Hierarchical clustering is a popular unsupervised data analysis method. For many real-world applications, we would like to exploit prior information about the data that imposes constraints on the clustering hierarchy, and is not captured by the set of features available to the algorithm. This gives rise to the problem …

2018-05-24abs ↗pdf ↗

Decision making is a process that is extremely prone to different biases. In this paper we consider learning fair representations that aim at removing nuisance (sensitive) information from the decision process. For this purpose, we propose to use deep generative modeling and adapt a hierarchical Variational Auto-Encode…

2018-06-26abs ↗pdf ↗

A novel hierarchical Bayesian approach to Federated Learning reduces data exposure and improves convergence rates.

problem Data privacy and convergence in Federated Learning.
method Hierarchical Bayesian modeling and block-coordinate descent optimization.
result The proposed algorithm converges to an optimal solution with a rate of O(1/t)O(1/\sqrt{t}) and guarantees vanishing generalization error.

A theoretical framework for deep learning is proposed to explain its effectiveness.

problem Lack of a comprehensive theory explaining deep learning's effectiveness.
method Integrates three characteristics into a graphical model called neurashed.
result Explains common empirical patterns in deep learning and provides insights into regularization and elasticity.

This paper fortifies the recently introduced hierarchical-optimization recursive least squares (HO-RLS) against outliers which contaminate infrequently linear-regression models. Outliers are modeled as nuisance variables and are estimated together with the linear filter/system variables via a sparsity-inducing (non-)co…

2019-10-11abs ↗pdf ↗

HS improves tree-based models' accuracy and interpretability without changing their structure.

problem Overfitting in tree-based models.
method Hierarchical Shrinkage (HS) post-hoc algorithm that shrinks tree predictions towards ancestor means.
result HS significantly improves predictive performance and interpretability of decision trees and RFs.

In this paper, we present Gamma-LSTM, an enhanced long short term memory (LSTM) unit, to enable learning of hierarchical representations through multiple stages of temporal abstractions. Gamma memory, a hierarchical memory unit, forms the central memory of Gamma-LSTM with gates to regulate the information flow into var…

2019-10-25abs ↗pdf ↗

Inverse inference, or "brain reading", is a recent paradigm for analyzing functional magnetic resonance imaging (fMRI) data, based on pattern recognition and statistical learning. By predicting some cognitive variables related to brain activation maps, this approach aims at decoding brain activity. Inverse inference ta…

2011-05-02abs ↗pdf ↗

Hierarchical clustering is a class of algorithms that seeks to build a hierarchy of clusters. It has been the dominant approach to constructing embedded classification schemes since it outputs dendrograms, which capture the hierarchical relationship among members at all levels of granularity, simultaneously. Being gree…

2018-06-09abs ↗pdf ↗

Variable selection for models including interactions between explanatory variables often needs to obey certain hierarchical constraints. The weak or strong structural hierarchy requires that the existence of an interaction term implies at least one or both associated main effects to be present in the model. Lately, thi…

2014-11-17abs ↗pdf ↗

New method automates asymmetric choice for better skill transfer in reinforcement learning.

problem Improving sample efficiency and transferability of reinforcement learning agents.
method Attentive Priors for Expressive and Transferable Skills (APES) using hierarchical KL-regularization.
result APES automates asymmetric choice, leading to better skill transfer across sequential tasks.

We construct a deep portfolio theory. By building on Markowitz's classic risk-return trade-off, we develop a self-contained four-step routine of encode, calibrate, validate and verify to formulate an automated and general portfolio selection process. At the heart of our algorithm are deep hierarchical compositions of p…

2016-05-23abs ↗pdf ↗