New framework for dense weighted networks with community-specific patterns.
problem Dense networks with varying edge weights across communities.
method Proposes a new model with functions mapping node characteristics to edge weights, requiring few parameters.
result Developed a bootstrap methodology for generating new networks.
Dense neural networks can't approximate all functions.
problem Approximation capabilities of dense neural networks.
method Model compression approach combining weak regularity lemma and graph neural networks.
result Existence of Lipschitz continuous functions not approximable by dense neural networks.
This paper improves speech recognition models using deep neural networks.
problem Improving automatic speech recognition models.
method Deployment of residual, densely-connected, and highway networks into speech recognition.
result Significant improvements in speech recognition models.
Paper proposes dense average network for improved power load forecasting.
problem Improving power load forecasting accuracy to save millions for the power industry.
method Introduces dense average connection and constructs dense average network for power load forecasting.
result Proposed model outperforms existing methods on public datasets.
DenseNets improve accuracy and efficiency in convolutional networks.
problem Improving accuracy and efficiency in deep convolutional networks.
method Introducing Dense Convolutional Networks (DenseNet) with direct connections between all layers.
result DenseNets achieve significant improvements over state-of-the-art networks on object recognition benchmarks.
New models explain residual and dilated dense neural networks using sparse coding.
problem Lack of theoretical understanding of residual and dilated dense neural networks.
method Proposed Res-CSC and MSD-CSC models, derived mathematical relationships, implemented ISTA.
result Mathematical understanding of residual and dilated dense neural networks.
DVNet efficiently segments large neurovascular datasets using skip connections.
problem Challenges in segmenting large neurovascular datasets from high-throughput microscopy data.
method A fully-convolutional, deep, and densely-connected encoder-decoder network with skip connections.
result DVNet achieves superior performance in semantic segmentation of neurovascular datasets.
New neuron model learns and adapts its receptive field.
problem Learning and focusing on informative inputs.
method Adaptive locally connected neuron model using backpropagation.
result Focusing neurons outperform dense layers in classification tasks.
New method computes dense partial correlations with applications in graph theory and uncertainty quantification.
problem Sparse inverse covariance matrices are popular but dense solutions are overlooked.
method Derives approach based on inverse problem theory.
result New insights and approaches for model selection and data preprocessing.
RDL-Net improves speech enhancement with fewer parameters and better performance.
problem Improving speech enhancement with fewer parameters and better performance.
method Proposes RDL-Net, a CNN combining residual and dense aggregations without over-allocating parameters.
result RDL-Net achieves higher speech enhancement performance with fewer parameters and lower computational requirements.
Detects anomalous behavior in social media users by analyzing content and connections.
problem Identifying disruptive patterns in user behavior on social media platforms.
method Joint representation learning of content and connection to detect anomalous behavior.
result Observed densely connected users engaging in local politics and exhibiting troll-like behavior.
Sparse Transformers can approximate dense Transformers with only O(n) connections.
problem Can sparse Transformers approximate arbitrary sequence-to-sequence functions?
method Proposed sufficient conditions for universal approximation and proved that sparse Transformers with O(n) connections can approximate dense models.
result Sparse Transformers with O(n) connections can approximate the same function class as dense models with n^2 connections.
A new deep learning method for tissue-cleared image registration.
problem Efficient registration of high-resolution tissue-cleared images.
method Densely connected convolutional architecture for deformable image registration, unsupervised training.
result Comparable and superior registration performance to state-of-the-art methods, especially at higher resolutions.
New algorithms reduce communication in GNN training.
problem Higher communication costs in GNNs due to sparse connectivity.
method Parallel algorithms for sparse-dense matrix multiplication.
result Asymptotic reduction in communication compared to previous methods.
Sparse neural networks can improve performance with less memory.
problem Lack of fast memory limits deep neural network performance.
method Experimented with sparse neural network topologies, including pruning-based and RadiX-Nets.
result Sparse networks achieve comparable accuracy to dense networks but suffer instability at extreme sparsity.
Improved student networks boost image accuracy by 10% with same inference time.
problem Improving neural network efficiency on general hardware.
method Attention transfer and channel saliency metrics combined with runtime performance observations.
result Performance-enhanced student networks achieve up to 10% higher ImageNet accuracy.
Finding "densely connected clusters" in a graph is in general an important and well studied problem in the literature \cite{Schaeffer}. It has various applications in pattern recognition, social networking and data mining \cite{Duda,Mishra}. Recently, Ames and Vavasis have suggested a novel method for finding cliques i…
Deep neural networks improve modulation recognition accuracy.
problem Improving modulation recognition accuracy in wireless signals.
method Developed and tested deep neural network architectures, including CNN, ResNet, DenseNet, and CLDNN.
result Achieved high accuracy (up to 88.5%) in recognizing wireless signal modulations.
Binary neural networks trained from scratch achieve state-of-the-art results.
problem Training accurate binary neural networks from scratch is challenging.
method No prior knowledge and simple training strategy used.
result Achieved state-of-the-art results on standard benchmark datasets.
Paper develops Dense NN models for temporal-spatial data with improved performance.
problem Improving predictive performance and robustness in temporal-spatial modeling.
method Fully connected neural networks with ReLU activation, non-asymptotic bounds, manifold modeling, short-range dependence.
result Demonstrates superior performance in temporal-spatial modeling across various synthetic functions.
Classifies manifolds with dense conjugacy classes in their mapping class groups.
problem Classifying manifolds based on conjugacy classes in their mapping class groups.
method Analyzing connected orientable 2-manifolds and their mapping class groups.
result Mapping class groups of certain manifolds have dense conjugacy classes.
New pruning technique reduces model size by 10x with minimal accuracy loss.
problem Model over-parameterization and energy efficiency in resource-constrained environments.
method Gradual pruning technique applied across various models and datasets.
result Large-sparse models outperform small-dense models with up to 10x reduction in non-zero parameters.
MS-FCN improves cardiac left ventricle segmentation accuracy.
problem Accurate segmentation of left ventricle from MRI images.
method Multi-scale fully convolutional network with dense connectivity.
result MS-FCN achieves state-of-the-art Dice scores (0.93 on endocardium, 0.96 on epicardium).
We discuss dense embeddings of surface groups and fully residually free groups in topological groups. We show that a compact topological group contains a nonabelian dense free group of finite rank if and only if it contains a dense surface group. Also, we obtain a characterization of those Lie groups which admit a dens…
Develops a novel method to estimate non-Gaussian hydraulic conductivities efficiently.
problem Estimation of non-Gaussian hydraulic conductivity fields in subsurface flow models.
method Integrates adversarial autoencoders with residual dense convolutional networks for parameterization and surrogate modeling.
result Significantly reduces computation time for accurate inversion results.
Lottery tickets find good initializations for IMP with sparse training.
problem Finding good initializations for iterative magnitude pruning (IMP) in sparse networks.
method Empirical study of IMP performance with varying pre-training data and iterations.
result Training on a small fraction of data suffices to obtain good initializations for IMP.
Horizon saddle connections imply dense hyperbolic geodesics on dilation surfaces.
problem Characterize dilation surfaces with dense hyperbolic geodesics.
method Analyzing saddle connections and directional flow properties.
result Dilation surfaces with horizon saddle connections have dense hyperbolic geodesics.
Bi-Lipschitz rigidity theorem for dense subgroups of algebraic groups.
problem Characterizing dense subgroups of algebraic groups.
method Bi-Lipschitz rigidity theorem for Zariski dense discrete subgroups.
result No C1-smooth slim limit set for higher rank semisimple algebraic groups. The key idea of current deep learning methods for dense prediction is to apply a model on a regular patch centered on each pixel to make pixel-wise predictions. These methods are limited in the sense that the patches are determined by network architecture instead of learned from data. In this work, we propose the dense…
The study finds a theoretical bound for pre-training iterations needed for pruning to yield good subnetwork performance.
problem Discovering efficient subnetworks within pre-trained dense networks.
method Mathematical analysis of a two-layer, fully-connected network, validating with a multi-layer perceptron trained on MNIST.
result A logarithmically dependent threshold on dataset size for successful pruning.
This paper explores loss landscapes of sparse neural networks, finding unique characteristics compared to dense networks.
problem Understanding the loss landscape of sparse neural networks, especially one-hidden-layer networks.
method Analyzes sparse networks with dense and sparse final layers, focusing on linear and non-linear models.
result Sparse networks can have no spurious valleys under certain conditions, but spurious valleys and minima can exist for wide sparse networks.
This paper introduces a hierarchical associative memory model with multiple layers.
problem Limitations of traditional associative memory models with only one hidden layer.
method Develops a fully recurrent model with arbitrary layers, including locally connected ones, and a corresponding energy function.
result The model can dynamically assemble memories using weights from lower layers and higher layers' rules.
Neural networks are dense among Lipschitz functions with fixed Lipschitz constant.
problem Characterizing neural network approximations to Lipschitz functions.
method Analyzing L-Lipschitz neural networks and their density in L-Lipschitz functions. result One layer neural networks are dense in the set of all L-Lipschitz functions. We show that for every sequence (ni), where each ni is either an integer greater than 1 or is ∞, there exists a simply connected open 3-manifold M with a countable dense set of ends {ei} so that, for every i, the genus of end ei is equal to ni. In addition, the genus of the ends not in the d…
Improves audio source separation using dilated convolutions and dense connections.
problem Optimizing feature extraction in audio source separation models.
method Adaptive dilated convolutions and dense connections in U-Net architecture.
result Improved performance on MUSDB test dataset.
Capsule Networks improve clothing retrieval without landmark info.
problem In-shop clothing retrieval performance improvement.
method Triplet-based Capsule Network architecture with SC and RC blocks.
result Triplet Capsule Networks outperform FashionNet and SOTA architectures.
Sparse sampling method for tensor factorization and completion of high rank tensors.
problem Completion of high rank tensors with missing data in recommendation systems.
method Sparse measurements and message-passing algorithms in a high-dimensional limit.
result Theoretical insights and performance analysis of tensor factorization in dense limit.
Novel graph theory for neural networks improves understanding of their structure and performance.
problem Understanding the structural benefits and generalization power of neural networks.
method Developed a novel graph theoretical formulation and extended error analysis for neural networks.
result Similar a priori estimates can be obtained for neural networks under certain conditions, independent of input dimension.
Enhanced image denoising with MWRDCNN using residual dense blocks.
problem Image denoising with improved performance and robustness.
method Multi-wavelet residual dense convolutional neural network (MWRDCNN) with residual dense blocks (RDBs).
result Significantly improved performance in image denoising compared to existing techniques.
Currents on cusped hyperbolic surfaces have a denseness property similar to compact surfaces.
problem Proving denseness of rational currents on cusped hyperbolic surfaces.
method Using geodesic currents and subset currents, proving denseness through examples and continuous extension.
result Denseness of rational currents on cusped hyperbolic surfaces, including geodesics connecting cusps.
Single neurons can perform as well as dense networks in binary and multi-class recognition tasks.
problem Designing efficient neural networks for recognition tasks.
method Investigated the use of single or multiple neurons in neural networks for binary and multi-class recognition tasks.
result Sparse networks can be as efficient as dense networks in both binary and multi-class tasks.
Generative model captures hubs and dense communities in social networks.
problem Capturing both hubs and dense communities in social networks.
method Graphon mixture model with a new condition on sparse graphs.
result Estimation of hub normalized degree and graphon for sparse components.
New method selects neural network architectures without needing data.
problem Choosing efficient deep neural network architectures.
method Developed the deep frame potential to quantify network capacity.
result Deep frame potential correlates with generalization error.
Method generates dense fields from sparse measurements without needing spatial statistics or examples.
problem Generating dense physical fields from sparse measurements.
method Introduces a differentiable numerical simulator into neural network training.
result Superior results on fluid mechanics problems compared to statistical and neural network methods.
Method finds interestingly dense subgroup connections in graphs.
problem Understanding patterns in graph connectivity based on node attributes.
method Information-theoretic definition of interestingness for subgroup connections.
result Identifies pairs of node subgroups with high or low edge density.
Sparse Hopfield model improves memory retrieval with fewer connections.
problem Memory retrieval efficiency with fewer connections.
method Sparse extension of Hopfield model, derived from sparse entropic regularizer.
result Sparse Hopfield model achieves tighter error bounds and better performance.
Polynomial-time test for detecting dense subgraphs in heterogeneous networks.
problem Detecting a planted community in heterogeneous networks.
method Proposes a polynomial-time test with a standard normal distribution null limiting distribution.
result The test is efficient and performs well in both simulations and real data.
New models capture heterogeneous network density, improving community detection.
problem Empirical networks are often globally sparse but locally dense.
method Latent Poisson models generating hidden multigraphs.
result These models improve community detection in sparse networks.