We present convolutional neural network (CNN) based approaches for unsupervised multimodal subspace clustering. The proposed framework consists of three main stages - multimodal encoder, self-expressive layer, and multimodal decoder. The encoder takes multimodal data as input and fuses them to a latent space representa…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Self-expressive Layer”
RED-SC improves deep subspace clustering efficiency.
problem Slow convergence in training deep subspace clustering models.
method Residual Encoder-Decoder network with skip-layer connections and self-expressive layer.
result Training converges much faster with RED-SC.
We propose Deep Closed-Form Subspace Clustering (DCFSC), a new embarrassingly simple model for subspace clustering with learning non-linear mapping. Compared with the previous deep subspace clustering (DSC) techniques, our DCFSC does not have any parameters at all for the self-expressive layer. Instead, DCFSC utilizes …
Data-aware methods for dimensionality reduction and matrix decomposition aim to find low-dimensional structure in a collection of data. Classical approaches discover such structure by learning a basis that can efficiently express the collection. Recently, "self expression", the idea of using a small subset of data vect…
A novel unsupervised feature selection method using subspace clustering and self-expressive model.
problem Feature selection for large datasets with minimal labeling effort.
method Subspace clustering with adaptive representation learning and regularized regression.
result The method effectively captures sample similarities and discriminative information.
New method learns subspace affinity function for scalable and inductive clustering.
problem Scaling and inductiveness of self-expressiveness methods for high-dimensional data.
method Siamese neural network architecture for metric learning.
result Model scales to larger datasets and clusters out-of-sample data.
Paper proposes a deep subspace clustering method using multi-level representations.
problem Deep subspace clustering of images.
method Convolutional autoencoders with multiple fully-connected layers for multi-level representations, loss minimization with iterative updates.
result The method outperforms state-of-the-art methods on real-world datasets.
A framework for clustering evolving high-dimensional data using LSTM networks.
problem Clustering evolving high-dimensional data with temporal evolution patterns.
method LSTM-ESCM framework exploiting self-expressive trait and LSTM networks.
result The proposed algorithm outperforms other methods in terms of run time and accuracy.
DS2CF-Net learns hierarchical representations with deep coupled factorization and enriched prior.
problem Learning deep hierarchical representations from data.
method Dual-constrained Deep Semi-Supervised Coupled Factorization Network (DS2CF-Net) with enriched prior.
result DS2CF-Net achieves state-of-the-art performance in representation learning and clustering.
Subspace segmentation or subspace learning is a challenging and complicated task in machine learning. This paper builds a primary frame and solid theoretical bases for the minimal subspace segmentation (MSS) of finite samples. Existence and conditional uniqueness of MSS are discussed with conditions generally satisfied…
DSCF-Net learns deep features for clustering with robustness and locality preservation.
problem Unsupervised deep representation learning for clustering.
method Integrates robust deep concept factorization, deep self-expressive representation, and adaptive locality preserving feature learning.
result Delivers state-of-the-art performance on public databases.
Stochastic Sparse Subspace Clustering improves subspace clustering by reducing over-segmentation through dropout.
problem Over-segmentation in subspace clustering.
method Introducing dropout regularization to enforce denser connections between points from the same subspace.
result Stochastic Sparse Subspace Clustering effectively handles large datasets and reduces over-segmentation.
Proposes a graph learning framework for clustering and semi-supervised classification.
problem Graph-based clustering and semi-supervised classification techniques have shown impressive performance but lack explicit cluster structure.
method Uses self-expressiveness of samples for global structure and adaptive neighbor approach for local structure. Incorporates rank constraint to ensure optimal performance.
result The proposed method achieves better performance than state-of-the-art methods in clustering and semi-supervised classification.
Develops methods for causal inference in longitudinal data.
problem Estimating Individual Treatment Effects (ITEs) in high-dimensional, time-varying data.
method Causal Dynamic Variational Autoencoder (CDVAE) and long-term counterfactual regression framework.
result CDVAE outperforms baselines and improves state-of-the-art models, approaching oracle performance.
Sparse Subspace Clustering (SSC) is a popular unsupervised machine learning method for clustering data lying close to an unknown union of low-dimensional linear subspaces; a problem with numerous applications in pattern recognition and computer vision. Even though the behavior of SSC for complete data is by now well-un…
A Novel Efficient Approach with Data-Adaptive Capability for OMP-based Sparse Subspace Clusteringcs.LG
Orthogonal Matching Pursuit (OMP) plays an important role in data science and its applications such as sparse subspace clustering and image processing. However, the existing OMP-based approaches lack of data adaptiveness so that the data cannot be represented well enough and may lose the accuracy. This paper proposes a…
Data similarity is a key concept in many data-driven applications. Many algorithms are sensitive to similarity measures. To tackle this fundamental problem, automatically learning of similarity information from data via self-expression has been developed and successfully applied in various models, such as low-rank repr…
Paper proposes a method to learn structure from data similarities.
problem Learning hidden structures in data beyond low-rank and sparsity.
method Explicitly models data relation, retaining pairwise similarities.
result Framework significantly improves performance on clustering and semisupervised classification.
Ankle sprains and instability are major public health concerns. Up to 70% of individuals do not fully recover from a single ankle sprain and eventually develop chronic ankle instability (CAI). The diagnosis of CAI has been mainly based on self-report rather than objective biomechanical measures. The goal of this study …
3-layer NTK models generalize better than 2-layer models, especially with large input dimensions.
problem Understanding the generalization of overparameterized neural networks.
method Analyzing the 3-layer NTK model's test error and comparing it to 2-layer NTK models.
result 3-layer NTK models have a faster descent in test error with respect to the number of neurons in the second hidden layer.
This paper presents two unsupervised learning layers (UL layers) for label-free video analysis: one for fully connected layers, and the other for convolutional ones. The proposed UL layers can play two roles: they can be the cost function layer for providing global training signal; meanwhile they can be added to any re…
Initialization Matters: On the Benign Overfitting of Two-Layer ReLU CNN with Fully Trainable Layerscs.LG
Two-layer CNNs can overfit well if initialized correctly.
problem Understanding the conditions for benign overfitting in over-parameterized CNNs.
method Extending analysis to fully trainable two-layer CNNs, examining initialization scaling effects.
result Initialization scaling of the output layer is crucial; large scales lead to fixed output behavior, small scales to complex interactions.
To infer a multilayer representation of high-dimensional count vectors, we propose the Poisson gamma belief network (PGBN) that factorizes each of its layers into the product of a connection weight matrix and the nonnegative real hidden units of the next layer. The PGBN's hidden layers are jointly trained with an upwar…
Proposes a fixed smooth convolutional layer to reduce checkerboard artifacts in CNNs.
problem Checkerboard artifacts in CNNs during upsampling and strided convolution.
method Fixed convolutional layer with adjustable smoothness, applied to four CNNs and GANs.
result Significantly improves classification performance and image generation quality.
Self-attention can replace convolutional layers in vision tasks.
problem The supremacy of convolutional layers in vision tasks.
method Analysis and experiments with self-attention layers compared to convolutional layers.
result Self-attention layers can perform as well as convolutional layers and learn to do so.
Three-layer networks learn more complex features than two-layer networks.
problem Understanding feature learning in deep neural networks.
method Analysis of three-layer neural networks trained with gradient descent.
result Three-layer networks can learn functions that two-layer networks cannot.
Gamma Belief Networksstat.ML
To infer multilayer deep representations of high-dimensional discrete and nonnegative real vectors, we propose an augmentable gamma belief network (GBN) that factorizes each of its hidden layers into the product of a sparse connection weight matrix and the nonnegative real hidden units of the next layer. The GBN's hidd…
Study shows LLMs can remove half of layers without significant performance drop.
problem Understanding knowledge storage in LLMs' weights.
method Layer pruning and finetuning to identify and remove unnecessary parameters.
result Minimal degradation of performance after removing up to half of layers.
The AIBC is an Artificial Intelligence and blockchain technology based large-scale decentralized ecosystem that allows system-wide low-cost sharing of computing and storage resources. The AIBC consists of four layers: a fundamental layer, a resource layer, an application layer, and an ecosystem layer. The AIBC implemen…
We improve neural network explainability by bypassing batch normalization.
problem Lack of transparency in neural networks.
method Layer-wise Relevance Propagation with a method to include normalization layers.
result Heatmaps are more accurate for convolutional layers with our method.
We address the problem of learning hierarchical deep neural network policies for reinforcement learning. In contrast to methods that explicitly restrict or cripple lower layers of a hierarchy to force them to use higher-level modulating signals, each layer in our framework is trained to directly solve the task, but acq…
Proposes continuous convolution layers for flexible feature map resizing.
problem Fixed stride limitations in discrete convolution layers.
method Introduces Continuous Convolution (CC) layers that use learned continuous functions.
result Dynamic and consistent resizing of feature maps at any scale, non-integer and axis-dependent.
One-layer transformers can't solve induction heads task efficiently.
problem Solving the induction heads task efficiently with one-layer transformers.
method Communication complexity argument showing exponential size requirement.
result No one-layer transformer can solve the induction heads task efficiently.
Unified framework LPCD optimizes quantization of complex submodules.
problem Quantization of complex submodules in neural networks.
method Layer-Projected Coordinate Descent (LPCD) for quantizing arbitrary submodules.
result LPCD enhances both layer-wise PTQ methods and existing submodule approaches.
A new neural network model MDRBM improves noise-robustness in classification.
problem Improving noise-robustness in classification problems.
method Stacking a probabilistic-ELM layer on a discriminative restricted Boltzmann machine (DRBM).
result MDRBM outperforms other models, especially in noise-robustness.
Strength of weak layers in cascading failures on multiplex networks: case of the international trade networkphysics.soc-ph
Many real-world complex systems across natural, social, and economical domains consist of manifold layers to form multiplex networks. The multiple network layers give rise to nonlinear effect for the emergent dynamics of systems. Especially, weak layers that can potentially play significant role in amplifying the vulne…
Network Implosion: Effective Model Compression for ResNets via Static Layer Pruning and Retrainingcs.LG
Residual Networks with convolutional layers are widely used in the field of machine learning. Since they effectively extract features from input data by stacking multiple layers, they can achieve high accuracy in many applications. However, the stacking of many layers raises their computation costs. To address this pro…
The study shows removing fully connected output layers improves efficiency without sacrificing performance.
problem Large number of parameters in fully connected layers for high-category datasets.
method Examined architectures replacing fully connected output layers with fixed layers and compared performance.
result Fixed classifiers offer no additional benefit over removing the output layer and its parameters.
Our work presents extensive empirical evidence that layer rotation, i.e. the evolution across training of the cosine distance between each layer's weight vector and its initialization, constitutes an impressively consistent indicator of generalization performance. In particular, larger cosine distances between final an…
Cross-Layer Subspace Coupling for LLM Compression: A Unifying Framework and Its Empirical Limitscs.LG
Unified SVD compression fails in practical tasks, highlighting the importance of per layer activation reconstruction.
problem The failure of a unified SVD compression method in practical tasks like perplexity and accuracy.
method Unified optimization problem for SVD based compression methods, focusing on cross-layer coupling.
result Downstream metrics like perplexity and accuracy degrade severely compared to standard per layer SVD LLM.
Introduces Causal Energy Minimization to understand Transformer layers.
problem Empirical parameterization of Transformer blocks remains largely unexplored.
method Causal Energy Minimization framework that recasts Transformer layers as optimization steps on conditional energy functions.
result Identifies design space for Transformer layers including weight sharing and energy-based interpretations.
A deep convolutional fuzzy system (DCFS) on a high-dimensional input space is a multi-layer connection of many low-dimensional fuzzy systems, where the input variables to the low-dimensional fuzzy systems are selected through a moving window across the input spaces of the layers. To design the DCFS based on input-outpu…
The rapid development of high-throughput technologies has enabled the generation of data from biological or disease processes that span multiple layers, like genomic, proteomic or metabolomic data, and further pertain to multiple sources, like disease subtypes or experimental conditions. In this work, we propose a gene…
PLN-Nets with two linear layers and parallel LN achieve universal approximation.
problem Limitations of standard neural network architectures in universal approximation.
method Introduced PLN-Nets combining two linear layers with parallel LN.
result PLN-Nets achieve universal approximation, while standard LN has limited power.
Smectic liquid crystals are materials formed by stacking deformable, fluid layers. Though smectics prefer to have flat, uniformly-spaced layers, boundary conditions can impose curvature on the layers. Since the layer spacing and curvature are intertwined, the problem of finding minimal configurations for the layers bec…
Improved recommendation systems using multi-layer embeddings reduce model size while maintaining accuracy.
problem Improving model accuracy in recommendation systems while minimizing model size.
method Introducing a multi-layer embedding training (MLET) architecture that trains embeddings via a sequence of linear layers.
result Substantial advantages in model accuracy and memory footprint are achieved with reduced embedding dimensions.
Inferencing with network data necessitates the mapping of its nodes into a vector space, where the relationships are preserved. However, with multi-layered networks, where multiple types of relationships exist for the same set of nodes, it is crucial to exploit the information shared between layers, in addition to the …
Batch-normalization (BN) layers are thought to be an integrally important layer type in today's state-of-the-art deep convolutional neural networks for computer vision tasks such as classification and detection. However, BN layers introduce complexity and computational overheads that are highly undesirable for training…