Estimates multiple dependent Gaussian graphical models for gene expression data.
problem Dependence among gene expression data from different tissues and the whole body.
method Decomposes the problem into systemic and category-specific layers, estimates them jointly using graphical EM.
result Estimation consistency and selection sparsistency of the proposed estimator.
JD.com uses a new CNN model to improve ad click prediction.
problem Improving CTR prediction for ads with visual content.
method Proposes Category-specific CNN (CSCNN) to incorporate category knowledge early in the feature extraction process.
result CSCNN outperforms existing methods in CTR prediction.
The paper tackles confidence calibration for exploratory machine learning problems.
problem Difficulty in curating datasets and confusion about category validity.
method Introduces four new algorithms for category-specific confidence estimation, including kernel density ratios.
result Kernel density ratios provide a novel approach to confidence calibration, especially for exploratory problems.
A new method for optimizing functions with both categorical and continuous variables.
problem Optimizing functions with mixed categorical and continuous variables.
method Formulates as a multi-armed bandit problem, uses Thompson sampling, extends to batch BO.
result Proves sub-linear regret bounds and demonstrates effectiveness on various benchmarks.
New method visualizes brain activity changes over time.
problem Understanding representational dynamics in neural responses.
method Procrustes-aligned Multidimensional Scaling (pMDS) on RDM movies.
result Multidimensional scaling alignment captures representational dynamics.
TXtract extracts structured knowledge from thousands of product categories.
problem Extracting structured knowledge from diverse product categories in e-commerce.
method TXtract uses a taxonomy-aware model with category conditional self-attention and multi-task learning.
result TXtract outperforms state-of-the-art approaches by up to 10% in F1 and 15% in coverage across all categories.
KeypointNet learns 3D keypoints for object pose estimation without ground-truth.
problem Learning 3D keypoints for object pose estimation without manual annotations.
method End-to-end geometric reasoning framework to discover keypoints.
result End-to-end framework outperforms fully supervised baseline.
New method transplants specific neural networks to generic ones without training samples.
problem Learning massive tasks and categories requires collecting samples for all at once.
method Designs a functionally interpretable generic network and uses back-distillation for transplanting.
result Method without training samples outperforms with 100 samples.
Model reconstructs novel 3D shapes with a single prior image.
problem Generalizing single-view 3D reconstruction to new classes with limited data.
method Reframes reconstruction as refinement of a provided prior shape.
result Model reconstructs novel classes with limited training data.
Paper proposes a method to prune neural networks, reducing storage and computation costs.
problem Reduction of storage and computational costs for deep neural networks.
method Statistical analysis of component significance using F-statistic-based screening technique.
result Pruned models are highly competitive with state-of-the-art approaches.
Bayesian model for discrete data with conditional transformations.
problem Handling discrete ordinal and count data with excess zeros.
method Bayesian framework with conditional transformation functions and modular MCMC algorithm.
result Flexible modeling of linear and nonlinear covariate effects for ordinal and count data.
GAME improves matrix completion by considering subgroup-specific latent structures.
problem Heterogeneous data with overlapping categories, smoothing away subgroup-specific variation.
method Group-Aware Matrix Estimation (GAME) with overlapping nuclear-norm penalties.
result GAME outperforms global low-rank estimators in structured missingness regimes.
Develops a new volatility model for prediction markets.
problem Volatility forecasting in prediction markets differs from standard asset markets.
method Combines Wright-Fisher and Glosten-Milgrom mechanisms to model binary prediction markets.
result Structural model outperforms standard ARCH/GARCH models in volatility forecasting.
Develops a new volatility model for prediction markets.
problem Volatility forecasting in prediction markets differs from standard asset markets.
method Combines Wright-Fisher and Glosten-Milgrom mechanisms to model binary prediction markets.
result Structural model outperforms standard ARCH/GARCH models in volatility forecasting.
End-to-end CAD system for thyroid nodule classification using multimodal data and expert guidance.
problem Improving accuracy in thyroid nodule classification for clinicians.
method Knowledge-driven DenseNet framework using multimodal ultrasound data and expert cues.
result The proposed system achieves relevant performances in thyroid nodule classification.
Adversarial MoE learns category-specific models for product search.
problem Variations in product features and importance across categories.
method Mixture of Experts with adversarial regularization and soft gating constraints.
result Improved clustering of gate output vectors and shared experts among similar categories.
3-layer NTK models generalize better than 2-layer models, especially with large input dimensions.
problem Understanding the generalization of overparameterized neural networks.
method Analyzing the 3-layer NTK model's test error and comparing it to 2-layer NTK models.
result 3-layer NTK models have a faster descent in test error with respect to the number of neurons in the second hidden layer.
Paper introduces UL layers for unsupervised video analysis.
problem Label-free video analysis.
method Two unsupervised learning layers for fully connected and convolutional neural networks.
result Neural networks with UL layers can extract shape and motion information from unlabeled videos.
Two-layer CNNs can overfit well if initialized correctly.
problem Understanding the conditions for benign overfitting in over-parameterized CNNs.
method Extending analysis to fully trainable two-layer CNNs, examining initialization scaling effects.
result Initialization scaling of the output layer is crucial; large scales lead to fixed output behavior, small scales to complex interactions.
Convolutional layers can be mathematically equated to fully connected layers.
problem Understanding the equivalence between convolutional and fully connected layers for neural networks.
method Demonstrated that convolutional operations can be converted to matrix multiplication, showing equivalence.
result Convolutional layers and fully connected layers are mathematically equivalent in linear cases.
Network Implosion reduces ResNet layers without accuracy loss.
problem High computation costs in Residual Networks.
method Static layer pruning and retraining to erase unimportant layers.
result Reduces ResNet layers by 24.00-42.86% without accuracy drop.
Proposes a fixed smooth convolutional layer to reduce checkerboard artifacts in CNNs.
problem Checkerboard artifacts in CNNs during upsampling and strided convolution.
method Fixed convolutional layer with adjustable smoothness, applied to four CNNs and GANs.
result Significantly improves classification performance and image generation quality.
Self-attention can replace convolutional layers in vision tasks.
problem The supremacy of convolutional layers in vision tasks.
method Analysis and experiments with self-attention layers compared to convolutional layers.
result Self-attention layers can perform as well as convolutional layers and learn to do so.
Three-layer networks learn more complex features than two-layer networks.
problem Understanding feature learning in deep neural networks.
method Analysis of three-layer neural networks trained with gradient descent.
result Three-layer networks can learn functions that two-layer networks cannot.
Study shows LLMs can remove half of layers without significant performance drop.
problem Understanding knowledge storage in LLMs' weights.
method Layer pruning and finetuning to identify and remove unnecessary parameters.
result Minimal degradation of performance after removing up to half of layers.
PGBN infers multilayer representations of count vectors using Gibbs sampling.
problem Inferring multilayer representations of high-dimensional count vectors.
method PGBN factorizes layers into product of weight matrices and hidden units, trained with Gibbs sampler.
result PGBN can add more layers to improve performance over Poisson factor analysis.
We improve neural network explainability by bypassing batch normalization.
problem Lack of transparency in neural networks.
method Layer-wise Relevance Propagation with a method to include normalization layers.
result Heatmaps are more accurate for convolutional layers with our method.
Model shows cascading failures are more severe in multiplex networks than single-layer networks.
problem Underestimation of risks in single-layer network analyses due to overlooked impact of weak layers.
method Simple model of cascading failure on multiplex networks of weight-heterogeneous layers.
result Multiplex model produces more catastrophic cascading failures than single-layer model.
Layer rotation predicts model generalization, improving test accuracy by up to 30%.
problem Predicting model generalization in deep networks.
method Monitoring the cosine distance between layer weights and their initial values during training.
result Training procedures that maximize layer rotation consistently lead to better generalization performance.
GBN infers deep representations from discrete vectors.
problem Inferring multilayer deep representations of high-dimensional data.
method GBN factorizes layers into sparse weight matrices and hidden units, trained with Gibbs sampler.
result GBN can improve performance in feature extraction and prediction.
MGCN improves multi-layer graph classification using node attributes and relations.
problem Lack of comprehensive multi-layer graph embedding methods considering node attributes and different types of edges.
method Proposes MGCN, a method that combines GCN for multi-layer graphs, incorporating node attributes and both within and between layer relations.
result MGCN outperforms other multi-layer and single-layer methods in semi-supervised node classification tasks.
A robot learns to classify images with limited perception using a layered reinforcement learning approach.
problem Image classification for robots with partial perception.
method Three-layer architecture using deep reinforcement learning, including meta-layer, action-layer, and classification-layer.
result The method achieves high accuracy on the MNIST dataset and provides explainability of the agent's decision-making process.
Proposes continuous convolution layers for flexible feature map resizing.
problem Fixed stride limitations in discrete convolution layers.
method Introduces Continuous Convolution (CC) layers that use learned continuous functions.
result Dynamic and consistent resizing of feature maps at any scale, non-integer and axis-dependent.
Lean 2-layer RBMs achieve similar representational power as single-layer RBMs with fewer parameters.
problem Understanding and quantifying the representational power of multi-layer RBMs.
method Inherent Structure Capacity (ISC) and Lean RBMs.
result 2-layer RBMs can achieve the same representational power as single-layer RBMs with fewer parameters.
Proposes a method for multi-layered network embeddings that improves performance.
problem Mapping multi-layered network nodes into a vector space while preserving relationships.
method Jointly embed nodes in all layers via DeepWalk on a supra graph, then fine-tune embeddings for cohesive structure.
result Outperforms existing single- and multi-layered network embedding algorithms on benchmarks.
Framework for integrating multi-layered data from multiple sources.
problem Integrating data from multiple layers and sources.
method Decompose multi-layer problem into two-layer problems, use neighborhood selection and group-penalized regression for sparse estimates, develop debiasing technique and testing procedures.
result Established global and simultaneous testing procedures for inter-layer directed edge weights.
Unified framework LPCD optimizes quantization of complex submodules.
problem Quantization of complex submodules in neural networks.
method Layer-Projected Coordinate Descent (LPCD) for quantizing arbitrary submodules.
result LPCD enhances both layer-wise PTQ methods and existing submodule approaches.
One-layer transformers can't solve induction heads task efficiently.
problem Solving the induction heads task efficiently with one-layer transformers.
method Communication complexity argument showing exponential size requirement.
result No one-layer transformer can solve the induction heads task efficiently.
AIBC uses AI and blockchain for efficient resource sharing.
problem Efficient resource sharing in large-scale decentralized systems.
method Two-consensus scheme with DABFT and DPoEV.
result Predictive deep learning for optimal BFT algorithm selection.
A new neural network model MDRBM improves noise-robustness in classification.
problem Improving noise-robustness in classification problems.
method Stacking a probabilistic-ELM layer on a discriminative restricted Boltzmann machine (DRBM).
result MDRBM outperforms other models, especially in noise-robustness.
New insights suggest larger early layers improve CNN performance.
problem Common assumption of monotonously increasing feature counts in CNNs is challenged.
method Used a skew normal distribution to investigate feature amounts in CNN layers.
result Architectures favoring larger early layers yield better accuracy.
The study shows removing fully connected output layers improves efficiency without sacrificing performance.
problem Large number of parameters in fully connected layers for high-category datasets.
method Examined architectures replacing fully connected output layers with fixed layers and compared performance.
result Fixed classifiers offer no additional benefit over removing the output layer and its parameters.
Enhances reinforcement learning with hierarchical policies using latent variables.
problem Improving performance in reinforcement learning tasks with hierarchical policies.
method Training each layer of a hierarchical neural network to solve tasks directly, with latent variables controlling lower layers.
result Improves performance on standard benchmark tasks and complex sparse-reward tasks.
Unified SVD compression fails in practical tasks, highlighting the importance of per layer activation reconstruction.
problem The failure of a unified SVD compression method in practical tasks like perplexity and accuracy.
method Unified optimization problem for SVD based compression methods, focusing on cross-layer coupling.
result Downstream metrics like perplexity and accuracy degrade severely compared to standard per layer SVD LLM.
Introduces Causal Energy Minimization to understand Transformer layers.
problem Empirical parameterization of Transformer blocks remains largely unexplored.
method Causal Energy Minimization framework that recasts Transformer layers as optimization steps on conditional energy functions.
result Identifies design space for Transformer layers including weight sharing and energy-based interpretations.
PLN-Nets with two linear layers and parallel LN achieve universal approximation.
problem Limitations of standard neural network architectures in universal approximation.
method Introduced PLN-Nets combining two linear layers with parallel LN.
result PLN-Nets achieve universal approximation, while standard LN has limited power.
Smectic liquid crystals are materials formed by stacking deformable, fluid layers. Though smectics prefer to have flat, uniformly-spaced layers, boundary conditions can impose curvature on the layers. Since the layer spacing and curvature are intertwined, the problem of finding minimal configurations for the layers bec…
Improved recommendation systems using multi-layer embeddings reduce model size while maintaining accuracy.
problem Improving model accuracy in recommendation systems while minimizing model size.
method Introducing a multi-layer embedding training (MLET) architecture that trains embeddings via a sequence of linear layers.
result Substantial advantages in model accuracy and memory footprint are achieved with reduced embedding dimensions.