A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Enhanced BCPNN learns hidden representations without labels.
DenseHMM improves HMMs by learning dense representations that enable gradient-based optimization.
We study how finite Bayesian neural networks adapt their hidden representations.
In this paper, we introduce transformations of deep rectifier networks, enabling the conversion of deep rectifier networks into shallow rectifier networks. We subsequently prove that any rectifier net of any depth can be represented by a maximum of a number of functions that can be realized by a shallow network with a …
Hidden symmetry of a G'-space X is defined by an extension of the G'-action on X to that of a group G containing G' as a subgroup. In this setting, we study the relationship between the three objects: (A) global analysis on X by using representations of G (hidden symmetry); (B) global analysis on X by using representat…
This paper aims to develop a new and robust approach to feature representation. Motivated by the success of Auto-Encoders, we first theoretical summarize the general properties of all algorithms that are based on traditional Auto-Encoders: 1) The reconstruction error of the input can not be lower than a lower bound, wh…
This work speeds up fHMM analysis by tensor algebra.
Revisits the connection between neural networks and the Kolmogorov-Arnold theorem.
Study reveals differences in medical image models' hidden representation refinement.
Spectral graph sparsification preserves geometry of GNN embeddings.
The autoencoder is an artificial neural network model that learns hidden representations of unlabeled data. With a linear transfer function it is similar to the principal component analysis (PCA). While both methods use weight vectors for linear transformations, the autoencoder does not come with any indication similar…
We train multi-task autoencoders on linguistic tasks and analyze the learned hidden sentence representations. The representations change significantly when translation and part-of-speech decoders are added. The more decoders a model employs, the better it clusters sentences according to their syntactic similarity, as t…
We explore and expand the to measure the of class manifolds in representation space: i.e., how close pairs of points from the same class are relative to pairs of points from different classes. We demonstrate several use cases of the loss. As an analytical to…
Study explores reinforcement learning in a complex game environment, analyzing rule inference and policy learning.
Spotlight method finds hidden errors in deep learning models.
Paper proposes a new model and methods for robustly de-interleaving HMP mixtures.
Deep Convolutional Neural Networks (CNN) enforces supervised information only at the output layer, and hidden layers are trained by back propagating the prediction error from the output layer without explicit supervision. We propose a supervised feature learning approach, Label Consistent Neural Network, which enforces…
Unsupervised Domain Adaptation aims to learn a model on a source domain with labeled data in order to perform well on unlabeled data of a target domain. Current approaches focus on learning \textit{Domain Invariant Representations}. It relies on the assumption that such representations are well-suited for learning the …
Wide neural networks' last hidden layers split into groups of redundant neurons.
Study shows label noise impacts neural representations' information content, revealing double descent behavior.
There is some theoretical evidence that deep neural networks with multiple hidden layers have a potential for more efficient representation of multidimensional mappings than shallow networks with a single hidden layer. The question is whether it is possible to exploit this theoretical advantage for finding such represe…
A neural network with a single hidden layer can't represent certain multivariable functions.
Unsupervised deep learning is one of the most powerful representation learning techniques. Restricted Boltzman machine, sparse coding, regularized auto-encoders, and convolutional neural networks are pioneering building blocks of deep learning. In this paper, we propose a new building block -- distributed random models…
Regularized training of an autoencoder typically results in hidden unit biases that take on large negative values. We show that negative biases are a natural result of using a hidden layer whose responsibility is to both represent the input data and act as a selection mechanism that ensures sparsity of the representati…
Debias recommender systems by accounting for hidden confounders using network information.
We present a novel neural network algorithm, the Tensor Switching (TS) network, which generalizes the Rectified Linear Unit (ReLU) nonlinearity to tensor-valued hidden units. The TS network copies its entire input vector to different locations in an expanded representation, with the location determined by its hidden un…
Kolmogorov neural networks can represent various types of functions.
Learning informative representations of data is one of the primary goals of deep learning, but there is still little understanding as to what representations a neural network actually learns. To better understand this, subspace match was recently proposed as a method for assessing the similarity of the representations …
We provide a classification of graphical models according to their representation as subfamilies of exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical models and chain graphs with no hidden variables, including Bayesian networks …
Study on hidden units in finite Bayesian neural networks and their tail properties.
CVAE learns disentangled and coupled representations without prior knowledge.
We present a variation of the Autoencoder (AE) that explicitly maximizes the mutual information between the input data and the hidden representation. The proposed model, the InfoMax Autoencoder (IMAE), by construction is able to learn a robust representation and good prototypes of the data. IMAE is compared both theore…
Deep learning has recently been shown to be instrumental in the problem of domain adaptation, where the goal is to learn a model on a target domain using a similar --but not identical-- source domain. The rationale for coupling both techniques is the possibility of extracting common concepts across domains. Considering…
New findings on hidden symmetries in ReLU networks.
This paper extends depth separation results to piece-wise oscillatory functions.
DNNs with regularization reveal feature learning dynamics and sparsity.
This paper continues our exploration of homology cobordism of 3-manifolds using our recent results on Cheeger-Gromov rho-invariants associated to amenable representations. We introduce a new type of torsion in 3-manifold groups we call hidden torsion, and an algebraic approximation we call local hidden torsion. We cons…
Deep networks have achieved impressive results across a variety of important tasks. However a known weakness is a failure to perform well when evaluated on data which differ from the training distribution, even if these differences are very small, as is the case with adversarial examples. We propose Fortified Networks,…
Deep neural networks excel at learning the training data, but often provide incorrect and confident predictions when evaluated on slightly different test examples. This includes distribution shifts, outliers, and adversarial examples. To address these issues, we propose Manifold Mixup, a simple regularizer that encoura…
Proposes DWMD for better matching of hidden representations across domains.
Recent results on optimization and generalization properties of neural networks showed that in a simple two-layer network, the alignment of the labels to the eigenvectors of the corresponding Gram matrix determines the convergence of the optimization during training. Such analyses also provide upper bounds on the gener…
Efficiently extracts linear dynamics from complex observations.
Semi-Non-negative Matrix Factorization is a technique that learns a low-dimensional representation of a dataset that lends itself to a clustering interpretation. It is possible that the mapping between this new representation and our original data matrix contains rather complex hierarchical information with implicit lo…
New method for causal effect estimation with hidden confounders.
We solve a high-dimensional model where nonlinear autoencoders detect hidden structure missed by PCA.
In this paper, we investigate the unsupervised deep representation learning issue and technically propose a novel framework called Deep Self-representative Concept Factorization Network (DSCF-Net), for clustering deep features. To improve the representation and clustering abilities, DSCF-Net explicitly considers discov…
Compressed LLM embeddings improve noisy regression tasks without overfitting.
This study investigates abrupt learning dynamics in Transformers, revealing plateau formation and internal representation collapse.