In this paper, we present a novel unsupervised feature learning architecture, which consists of a multi-clustering integration module and a variant of RBM termed multi-clustering integration RBM (MIRBM). In the multi-clustering integration module, we apply three unsupervised K-means, affinity propagation and spectral c…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “hidden features”
Study on generalisation in random feature learning and hidden manifold models.
problem Generalisation in high-dimensional learning problems.
method Replica method from statistical physics for asymptotic generalisation performance.
result Closed-form expression for generalisation performance in various high-dimensional settings.
We are often interested in explaining data through a set of hidden factors or features. When the number of hidden features is unknown, the Indian Buffet Process (IBP) is a nonparametric latent feature model that does not bound the number of active features in dataset. However, the IBP assumes that all latent features a…
Feature Learning for Nonlinear Dimensionality Reduction toward Maximal Extraction of Hidden Patternscs.LG
FEALM learns features for better nonlinear DR of hidden patterns.
problem DR misses important patterns on distorted manifolds.
method FEALM generates optimized projections using an optimization algorithm and neighbor-shape dissimilarity.
result FEALM captures important patterns on hidden manifolds.
Researchers calculate Shannon entropy rates of hidden Markov processes efficiently.
problem No finite expression exists for Shannon entropy rates of hidden Markov processes.
method Developed an efficient method to calculate entropy rates and identify minimal predictive features.
result Entropy rates can be accurately calculated for hidden Markov processes.
Deep Convolutional Neural Networks (CNN) enforces supervised information only at the output layer, and hidden layers are trained by back propagating the prediction error from the output layer without explicit supervision. We propose a supervised feature learning approach, Label Consistent Neural Network, which enforces…
Learning with hidden variablesq-bio.NC
Learning and inferring features that generate sensory input is a task continuously performed by cortex. In recent years, novel algorithms and learning rules have been proposed that allow neural network models to learn such features from natural images, written text, audio signals, etc. These networks usually involve de…
New framework identifies hidden risks and optionality in American options.
problem Underestimation of flexibility and convexity in early-exercise features.
method Introducing stochasticity into underlying determinants to quantify hidden risks and optionality.
result Remedies conventional pricing systems that underestimate optionality.
We study how finite Bayesian neural networks adapt their hidden representations.
problem Understanding how finite Bayesian neural networks differ from infinite ones.
method We analyze the asymptotics of learned feature kernels for various network architectures.
result The leading finite-width corrections to feature kernels have a universal form.
Self-regularizing restricted Boltzmann machinescond-mat.dis-nn
Self-regularizing RBMs learn optimal hidden units efficiently.
problem Learning optimal number of hidden units in RBMs.
method Grand-canonical extension of RBMs with varying hidden units, using chemical potential to control size.
result Efficiently deduces optimal number of hidden units with small generalization error.
New method discovers concepts in hidden feature layers using sparse subspace clustering.
problem Local attribution methods fail to identify coherent model behavior across samples.
method Sparse Subspace Clustering (SSCC) for concept discovery.
result Empirically validated method for various image classification tasks.
A new approach for feature extraction from time series is proposed in this paper. This approach consists of a specific regression model incorporating a discrete hidden logistic process. The model parameters are estimated by the maximum likelihood method performed by a dedicated Expectation Maximization (EM) algorithm. …
The problem of attributing a deep network's prediction to its \emph{input/base} features is well-studied. We introduce the notion of \emph{conductance} to extend the notion of attribution to the understanding the importance of \emph{hidden} units. Informally, the conductance of a hidden unit of a deep network is the \e…
Toward a Metrology for Artificial Intelligence: Hidden-Rule Environments and Reinforcement Learningcs.LG
Study explores reinforcement learning in a complex game environment, analyzing rule inference and policy learning.
problem Learning optimal policies in environments with hidden rules.
method Investigated using the Game Of Hidden Rules (GOHR) environment, employing Feature-Centric and Object-Centric state representations with a Transformer-based A2C algorithm.
result Transformer-based A2C models outperform traditional methods in GOHR, demonstrating the effectiveness of representation strategies.
LIFE framework improves model accuracy and interpretability.
problem Achieving high prediction accuracy and interpretability in neural networks.
method Three-step process: subset definition, feature creation, and linear model combination.
result LIFE consistently outperforms other models in prediction accuracy and interpretability.
This paper aims to develop a new and robust approach to feature representation. Motivated by the success of Auto-Encoders, we first theoretical summarize the general properties of all algorithms that are based on traditional Auto-Encoders: 1) The reconstruction error of the input can not be lower than a lower bound, wh…
A Neural Network for Determination of Latent Dimensionality in Nonnegative Matrix Factorizationcs.LG
A neural network method determines the latent dimensionality of NMF.
problem Determining the correct number of hidden features (latent dimensionality) in NMF.
method Combining NMFk with an MLP classifier trained on a dataset of matrices with known latent features.
result The MLP classifier in conjunction with NMFk achieves a greater than 95% success rate in determining the correct number of latent features.
Deep learning rules simplified with implicit methods.
problem Complexity and interpretability in deep learning models.
method Simplified notation through fixed-point equations and implicit features.
result Simplified deep learning models with enhanced interpretability.
A new method learns outcome-aware spectral features for causal effect estimation.
problem Estimation of causal effects in the presence of hidden confounders.
method Augmented Spectral Feature Learning framework that minimizes a contrastive loss derived from an augmented operator incorporating outcome information.
result Our method remains effective even under spectral misalignment.
DNNs with regularization reveal feature learning dynamics and sparsity.
problem Understanding feature learning in DNNs with regularization.
method Reformulating loss in terms of layerwise activations and covariances.
result Proving sparsity of local minima in -regularized DNNs.
Neural networks favor simple features over complex ones, even when complex features are available.
problem Neural networks exhibit a bias towards simple features over complex ones, even when complex features are present.
method Rigorously defined simplicity bias, theoretical and empirical demonstrations, ensemble approach to improve robustness.
result One hidden layer neural networks favor simple features over complex ones, even in the presence of more robust features.
We search for digital biomarkers from Parkinson's Disease by observing approximate repetitive patterns matching hypothesized step and stride periodic cycles. These observations were modeled as a cycle of hidden states with randomness allowing deviation from a canonical pattern of transitions and emissions, under the hy…
Supervised learning frequently boils down to determining hidden and bright parameters in a parameterized hypothesis space based on finite input-output samples. The hidden parameters determine the attributions of hidden predictors or the nonlinear mechanism of an estimator, while the bright parameters characterize how h…
DeepUnHide uses deep learning to reveal hidden demographic features in recommender systems.
problem Extracting hidden demographic features from recommender systems factors.
method Gradient-based localization in deep learning for feature extraction.
result DeepUnHide outperforms state-of-the-art feature selection methods.
Wider networks learn more interpretable features and improve performance during fine-tuning.
problem Transferability of learned features between tasks and the effect of network width on feature learning.
method Activation atlases to visualize and analyze features learned by wide and narrow networks.
result The hidden state of a wide network contains more information about the inputs than a narrow network, leading to improved performance during fine-tuning.
PyHHMM is a Python library for HHMMs with advanced features.
problem Handling heterogeneous observation models and missing data in HMMs.
method Object-oriented Python implementation with advanced features.
result PyHHMM supports a heterogeneous observation model and missing data inference.
DefogGAN: Predicting Hidden Information in the StarCraft Fog of War with Generative Adversarial Netscs.LG
DefogGAN predicts hidden RTS game information to aid strategic decision-making.
problem Predicting hidden information in real-time strategy games like StarCraft.
method Conditional Generative Adversarial Network (GAN) with pyramidal reconstruction loss.
result DefogGAN predicts enemy buildings and combat units as accurately as professional players.
New HMC method handles features in POS tagging, outperforming MEMM.
problem HMC struggles with arbitrary features in POS tagging.
method Introduced Entropic Forward-Backward (EFB) probabilities to compute HMC restorations.
result EFB-based HMC outperforms MEMM in POS tagging.
New framework analyzes deep neural networks using feature probabilities.
problem Degenerate situation in over-parameterized DNNs.
method Mean-field framework representing DNNs by feature probabilities and functions.
result Global convergence proof for over-parameterized Res-Net training.
Neural networks with random hidden nodes have gained increasing interest from researchers and practical applications. This is due to their unique features such as very fast training and universal approximation property. In these networks the weights and biases of hidden nodes determining the nonlinear feature mapping a…
New method models unknown systems with hidden parameters using neural networks.
problem Modeling unknown dynamical systems with hidden parameters.
method Training a deep neural network (DNN) model using trajectory data of the unknown system.
result DNN model accurately predicts unknown dynamical systems with new initial conditions.
Complex-Valued Restricted Boltzmann Machine for Direct Speech Parameterization from Complex Spectraeess.AS
This paper describes a novel energy-based probabilistic distribution that represents complex-valued data and explains how to apply it to direct feature extraction from complex-valued spectra. The proposed model, the complex-valued restricted Boltzmann machine (CRBM), is designed to deal with complex-valued visible unit…
Deep learning models misclassify malware with added benign features.
problem Detecting malware with deep learning when it's mixed with benign code.
method Trained a deep neural network classifier using benign and malware features. Demonstrated the impact of adding benign features to malware. Used data augmentation to improve classifier robustness.
result Adding benign features to malware significantly increases false negatives.
Proposes DWMD for better matching of hidden representations across domains.
problem Measuring data distribution discrepancy between semantically related domains for feature representation matching.
method DWMD, a moment-based probability distribution metric that explicitly orders and weights higher-order moments.
result DWMD is error-free and can strictly reflect distribution differences without feature distribution assumptions.
Flood extent mapping plays a crucial role in disaster management and national water forecasting. Unfortunately, traditional classification methods are often hampered by the existence of noise, obstacles and heterogeneity in spectral features as well as implicit anisotropic spatial dependency across class labels. In thi…
Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imagingcs.LG
Hidden stratification causes machine learning models to fail on rare but important patient subgroups.
problem Machine learning models fail on rare patient subgroups not identified during training or testing.
method Assessed techniques for measuring and describing hidden stratification effects on multiple medical imaging datasets.
result Evidence of hidden stratification leading to over 20% performance differences on clinically important subsets.
In Natural Language Processing (NLP) tasks, data often has the following two properties: First, data can be chopped into multi-views which has been successfully used for dimension reduction purposes. For example, in topic classification, every paper can be chopped into the title, the main text and the references. Howev…
DWTS uses observational data to improve clinical trial efficiency.
problem Lack of definitive conclusions from randomized clinical trials due to insufficient patient cohorts and confounding biases.
method DWTS combines observational data with randomized clinical trials using Doubly Debiased LASSO (DDL) to identify reliable covariates.
result DWTS reduces cumulative regret in clinical trials compared to standard methods.
New CFNN architecture approximates functions with machine accuracy.
problem Function approximation with high precision.
method Chebyshev Feature Neural Network (CFNN) with learnable frequencies.
result Achieves machine accuracy in function approximation.
While neural networks are powerful approximators used to classify or embed data into lower dimensional spaces, they are often regarded as black boxes with uninterpretable features. Here we propose Graph Spectral Regularization for making hidden layers more interpretable without significantly impacting performance on th…
Gamma Belief Networksstat.ML
To infer multilayer deep representations of high-dimensional discrete and nonnegative real vectors, we propose an augmentable gamma belief network (GBN) that factorizes each of its hidden layers into the product of a sparse connection weight matrix and the nonnegative real hidden units of the next layer. The GBN's hidd…
Student-teacher learning improves generalization with noisy inputs.
problem Transfer knowledge from clean inputs to noisy inputs.
method Analyzes student-teacher learning using deep linear networks and experiments with nonlinear networks.
result Three factors are vital for success: zero training loss, teacher knowledge, and feature decomposition.
Deep Stacked Stochastic Configuration Networks for Lifelong Learning of Non-Stationary Data Streamscs.LG
The concept of SCN offers a fast framework with universal approximation guarantee for lifelong learning of non-stationary data streams. Its adaptive scope selection property enables for proper random generation of hidden unit parameters advancing conventional randomized approaches constrained with a fixed scope of rand…
Neural networks outperform kernels by learning features better.
problem Current theories of feature learning do not adequately assess feature quality.
method Introduced feature quality metric and examined existing theories empirically.
result Current theories of feature learning do not provide a sufficient foundation for neural network generalization.
The financial crisis of 2008 generated interest in more transparent, rules-based strategies for portfolio construction, with Smart beta strategies emerging as a trend among institutional investors. While they perform well in the long run, these strategies often suffer from severe short-term drawdown (peak-to-trough dec…
Study reveals differences in medical image models' hidden representation refinement.
problem Understanding how intrinsic dimensionality changes in neural network hidden representations across different domains.
method Analysis of 11 natural and medical image datasets using 6 network architectures.
result Medical image models refine hidden representations earlier, suggesting differences in feature abstraction.
Our research focuses on analysing human activities according to a known behaviorist scenario, in case of noisy and high dimensional collected data. The data come from the monitoring of patients with dementia diseases by wearable cameras. We define a structural model of video recordings based on a Hidden Markov Model. N…
Theoretical study shows adversarial training improves robustness in deep learning models.
problem Ensuring robustness in pre-trained deep learning models.
method Theoretical analysis of adversarial training and feature purification in two-layer neural networks.
result Adversarial training leads to feature purification, making models more robust to attacks.